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PREFACE 


This  document  was  written  by  the  Systems  Reliability  and  Engineering  C  vision  of 
Rome  Air  Development  Center  as  an  “in-house"  project.  It  represents  th( 
collective  thoughts  and  experience  of  the  organization’s  over  30  years  in 
developing  and  applying  reliability  and  maintainability  principles.  Tfte  following 
individuals  were  the  primary  technical  contributors  to  the  completion  of  ti  e  project: 

Seymour  F.  Morris 
Preston  R.  MacDiarmid 
Anthony  J.  Feduccia 
Bruce  W.  Dudley 
Jerome  Klion 
David  Garafalo 

Ndi  icy  Phalen  was  responsible  for  the  typing  of  the  document. 

Comments  and/or  questions  relative  to  the  information  in  this  document  :an  be 
addressed  to; 

Systems  Reliability  &  Engineering  Division 
RADC/RBE 

Griffiss  AFB  NY  13441-5700 
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INTRODUCTION 


Purpose 

This  TOOLKIT  is  intended  for  use  by  a  practicing  reliability  and  niaintainability 
(R&M)  engineer.  Emphasis  is  placed  on  his  role  in  the  various  R&M  activities  of  an 
electronic  systems  development  program.  The  TOOLKIT  is  not  intended  to  be  a 
complete  tutorial  or  technical  treatment  of  the  R&M  discipline  but  rather  a 
compendium  of  useful  R&M  reference  information  to  be  used  in  everyday  practice. 

Format 

The  format  of  the  TOOLKIT  has  been  designed  for  easy  reference.  Five  main 
sections  are  laid  out  to  follow  the  normal  time  sequence  of  a  military  development 
program.  Each  section  contains  a  summary  cover  sheet  which  defines  the  process 
which  is  the  subject  of  the  section  and  describes  the  R&M  engineer’s  loie  in  the 
key  activities  of  that  process.  Because  the  processes  (sections)  are  interrelated, 
coverage  of  certain  steps  in  a  particular  process  may  be  by  reference  to  a 
different  section. 

Descriptions  of  the  “how  to"  of  the  R&M  engineer's  activities  have  been  designed 
to  take  the  form  of  figures,  tables,  and  step-by-step  procedures  as  opposed  to 
paragraphs  of  text.  Appendices  are  included  to  give  a  greater  depth  of  technical 
coverage  to  some  of  the  topics  as  well  as  to  present  additional  useful  reference 
information. 

The  TOOLKIT  also  includes  a  "Quick  Reference  Application  Index  '  which  can  be 
used  to  quickly  refer  the  R&M  engineer  to  the  portion  of  a  section  that  answers  his 
specific  questions.  A  quick  reference  "For  More  Help  Appendices"  index  is  also 
included  for  the  more  in-depth  topics  of  the  appendices. 

Terminology 

The  term  “Reliability"  used  in  the  title  of  this  document  is  used  in  the  broad  sense 
to  include  the  field  of  maintainability.  The  content  of  the  report  addresses  reliability 
and  maintainability  (R&M)  because  thev  are  usually  tfie  responsibility  of  one 
government  individual  in  a  military  electronics  development  program.  In  this 
context,  testability  is  considered  as  a  part  of  maintainability  and  is,  therefore, 
inherently  part  of  the  "M"  of  "R&M."  Where  testability  issues,  such  as  development 
of  quantitative  requirements,  are  appropriate  for  separation  from  "M  "  discussion, 
they  are  and  have  been  labeled  accordingly. 

Underlying  Philosophy 

The  development  and  application  of  a  successful  reliability  program  requires  a 
number  of  tasks  and  coordination  steps.  Key  ingredients  include; 

•  Aggressive  Program  Manager  Support  •  Thorough  Technical  Reviews 

•  Firm  and  Realistic  Requirements  •  Complete  Verification 

•  Effective  Built-in-Test 
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This  document  was  developed  to  support  this  R  ADC  philosophy  on  reliability 
and  maintainability 

R&M  2000 

Air  Force  leadership  is  committed  to  increasing  combat  capability  through 
improved  reliability  and  maintainability  (R&M).  In  September  1964,  the  Secretary 
and  Chief  of  Staff  of  the  Air  Force  stated  as  policy  that  R&M  would  be  equal  with 
cost,  schedule  and  (other)  performance  in  weapon  system  acquisition.  In  February 
1985,  the  Secretary'  and  Chief  of  Staff  promulgated  the  “R&M  2000  Action  Plan”  to 
guide  the  Air  Force  and  industry  institutionalization  process.  The  “R&M  2000 
Process"  contains  three  major  segments;  Goals  which  establish  the  basic 
purpose  and  direction  for  the  Air  Force  R&M  Program;  Principles  which  provide  a 
coherent,  logical  framework  for  many  actions  and  players  in  the  process;  and 
Building  Blocks  which  are  activities  that  will  result  in  a  successful  R&M  program. 
The  goals  can  be  achieved  by  application  of  the  building  blocks  within  the 
framework  of  the  principles. 


The  R&M  2000  Goals  are: 

1 .  Increase  Combat  Capability 

2  Decrease  the  Vulnerability  of  the  Combat  Support  Structure 
3.  Decrease  Mobility  Requirements  per  Unit 

4  Decrease  Manpower  Requirements  per  Unit  of  Output 

5  Decrease  Costs 


The  R&M  2000  process's  five  Principles  are: 

1  Management  Involvement 

2.  Motivation  of  Industry 

3.  Clearly  Communicated  Requirements 

4.  Designing  for  R&M  and  Grovirth  to  Inherent  R&M 

5.  Preservation  of  Inherent  R&M 
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The  R&M  2000  process  includes  21  Building  Blocks  as  follows. 


•  Source  Selection 

•  Performance  Based 
Progress 

•  Incentives  and 
Warranties 


n  and  Growth; 


•  Clear  Requirements 

•  Technician  Transparency 

•  Simplification 

•  Modularity 

•  R&M  Plans 

•  Company  Policies  and 
Practices 


Systems  Engineering 
Process 

Allocation  and  Prediction 
Analysis 

Growth  Management 
Parts  Selection 
Derating 

Computer  Aided  Tools 
Test-Analyze-and-Fix 


Variability  Reduction 
i^'ogram 

Environmental  Stress 
Screening 

System  Testing 

Feedback 


These  building  blocks  are  derived  from  examining  successful  programs;  they 
consist  of  time-tested  techniques  that  work. 

This  TOOLKIT,  although  not  structured  to  address  the  R&M  2000  building  blocks 
per  se,  addresses  the  practical  application  of  proven  reliability  and  maintainability 
techniques  that  results  in  meeting  f^e  R&M  2000  objectives. 
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Section  R 


How  do  I  develop  and  specify  the  right  RELIABILITY  & 
MAINTAINABILITY  requirements? 

R&M  Engineer’s  Role: 


Develop  Quantitative  Requirements 

R1  Reliability  .  13 

R2  Maintainability .  19 

R3  Testability .  22 

Develop  R&M  Task  Requirements 

R4  Program  Phase  Terminology .  24 

R5  R&M  Task  Appiication/Priority .  26 

R6  Specify  Contract  Data  Requirements .  28 

R7  Specify  Information  for  Proposals .  30 

R8  Estimate  Reliability  Program  Cost .  31 

SI  Develop  Proposal  Evaluation  Criteria .  33 

Appendix  2  Example  R&M  Program  Tasks . A>9 
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For  More  Information 


MIL-STD-470 

MILSTD-721 

MIL-STD-785 

MIL-STD-2165 
DOD  5010. 12-L 


“Maintainability  Program  for  Systems  and  Equipment” 

“Definition  of  Terms  for  Reliability  and  Maintainability" 

“Reliability  Program  for  Systems  and  Equipment  Development 
and  Production" 

“Testability  Programs  for  Electronic  Systems  and  Equipment” 

“Acquisition  Management  Systems  and  Data  Requirements 
Control  List"  (complete  Data  Item  Description  listing) 


AFR  ROO-2  “Acquisition  Program  Management” 

AFR  800-18  "Air  Force  Reliability  and  Maintainability  Program” 

RADC-TR-87-50  “R&M  Program  Cost  Drivers” 

"RADC  Program  Managers  Guide  to  Reliability  and 
Maintainability"  Slide  Rule  (order  directly  from  R  ADC/RBE, 
Griffiss  AFB  NY  13441-5700) 
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REQUIREMENTS— TOPIC  R1 


Scope  of  Roquiromonts: 

Reliability  parameters  expressed  by  operatiorial  users  and  ones  specified  in 
contractual  documents  take  many  forms.  TablesR1-1  and  R1-2  identify  the 
characteristics  of  reliability  parameters. 


Table  R1-1 :  Logistics  (Basic)  and  Mission 
Reliability  Characteristics 


Logistics  (Basic)  Reliability  Mission  Reliability 


•  Measure  of  system's  ability  to  operate 
without  logistics  support. 


*  Measure  of  system's  ability  to 
complete  mission. 


•  Recognize  effects  of  all  occurrences 
that  demand  support  without  regard  to 
effect  on  mission. 

•  Degraded  by  redundancy. 

•  Usually  equal  to  or  lower  than  mission 
reliability. 


•  Consider  only  failures  that  cause 
mission  abort. 

•  Improved  by  redundancy. 

•  Usually  higher  than  logistics 
reliability. 


Table  R1-2:  Operational  and  Contractual  Reliability 
Characteristics 


Contractual  Reliability 


•  Used  to  define,  measure  and  evaluate 
contractor's  program. 

•  Derived  from  operational  needs. 

•  Selected  such  that  achieving  them 
allows  projected  satisfaction  of 
operational  reliability. 

•  Expressed  in  inherent  values. 

•  Account  only  for  failure  events  subject 
to  contractor  control. 

•  Include  only  design  and  manufacturing 
characteristics. 


Operational  Reliability 


•  Used  to  describe  reliability 
performance  when  operated  in 
planned  environment. 

•  Not  used  for  contract  reliability 
requirements  (requires  translation). 

•  Used  to  describe  needed  level  of 
reliability  performance. 

•  Include  combined  effects  of  item 
design,  quality,  installation 
environment,  maintenance  policy, 
repair,  etc. 
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REQUIREMENTS— TOPIC  R1 


Table  R1-2  (continued) 

Contractual  Reliability 

Operational  Reliability 

•  Typical  terms; 

*  Typical  terms: 

•  MTBF  (mean-time-between-failures) 

•  MTBM  (mean-time-between- 

•  Mission  MTBF  (sometimes  also 

maintenance) 

called  MTBCF) 

•  MTBD  (mean-time-between- 
demand) 

•  MTBR  (mean-time-between- 
removal) 

•  MTBCF  (mean-time-between- 
critical-failure) 

Operational  Constraints: 

•  Mission  Criticality 

•  Availability  Constraints 

•  Self-Sufficiency  Constraints 

•  Attended/Unattended  Operation 

•  Operational  Environment 

•  Use  of  Off-the-shelf  or  Newly  Designed  Equipment 

How  to  Develop  Requirements: 

Figure  R1  -1  defines  the  general  reliability  requirement  development  process.  Key 
points  to  recognize  from  this  process  are; 

1 .  User  requirements  can  be  expressed  in  a  variety  of  forms  that  include 

combinations  of  mission  and  logistics  reliability,  or  they  may  combine  reliability 
with  maintainability  in  the  form  of  availability.  Conversion  to  commonly  used 
operational  terms  such  as  mean-time-between-maintenance  (MTBM)  and 
mean-time-between-critical-failure  (MTBCF)  must  be  made  from  terms  such  as 
operational  availability  (Ao),  and  break-rate,  etc.,  to  enable  translation  to 
parameters  which  can  be  specified  in  contracts. 

An  example  is; 

MTBM 

^  MTBM  +  MDT 

(Solve  for  MTBM  using  mean  downtime  (MDT)  which  includes  the  actual  repair 
time  plus  logistics  delay  time.) 
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2.  Since  operational  reliability  measures  take  into  account  factors  beyond  the 
control  of  development  contractors,  they  must  be  translated  to  contractual 
reliability  terms  for  which  contractors  can  be  held  accountable.  (Appendix  1 
provides  one  means  of  accomplishing  this  translation.) 

3.  The  process  cannot  end  with  the  translation  to  a  contractual  value.  Evaluation 
of  the  realism  of  the  translated  requirements  is  a  necessary  step.  Questions 
that  have  to  be  answered  are:  are  the  requirements  compatible  with  the 
available  technology  and  do  the  requirements  unnecessarily  drive  the  design 
(conflict  with  system  constraints  such  as  weight  and  power).  Addressing  these 
issues  requires  reviewing  previous  studies  and  data  for  similar  systems. 
Adjustment  factors  may  be  appropriate  for  improvement  of  technology  and  for 
different  operating  environments,  duty  cycles,  etc.  See  Topic  A1 1  for  Reliability 
Adjustment  Factors. 

4.  Systems  with  mission  critical  requirements  expressed  by  the  user  present 
difficulties  in  the  requirement  development  process.  Translation  models  don’t 
account  for  the  nonexponential  situations  that  exist  with  redundant  systems. 
Because  the  reliabilities  of  redundant  paths  are  high  compared  to  serial  ones, 
an  approximation  can  be  made  that  these  paths  have  an  equivalent  failure  rate 
of  zero  so  that  only  the  remaining  serial  elements  need  to  be  translated. 

5.  The  requirement  process  involves  allocation  of  values  to  lower  levels.  In  some 
cases,  this  is  an  iterative  process  requiring  several  tries  to  satisfy  all 
requirements.  For  other  cases,  the  requirements  can't  be  satisfied  and  dialogue 
and  tradeoffs  with  the  user  are  required. 

6.  For  cases  where  user  needs  are  not  specified  it  still  makes  sense  to  invoke  at 
least  a  logistics  (basic)  reliability  requirement.  In  so  doing,  the  contractor  has 
a  degree  of  accountability  and  is  likely  to  put  more  effort  into  designing  a 
reliable  system. 

7.  Table  R1  -3  indicates  typical  ranges  of  reliability  for  different  types  of 
electronic  systems: 
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Table  R1*3:  Typical  Reliability  Values 


Radar  Syatems  MTBF  (Hours) 

Ground  Rotating  Search  Radar . 75-175 

Large  Fixed  Phase  Array  Radar . 3-6 

Tactical  Ground  Mobile  Radar . 25-75 

Airborne  Fighter  Fire  CJontrol  Radar .  50-200 

Airborne  Search  Radar .  300-500 

Airborne  identification  Radar .  200-2,000 

Airborne  Navigation  Radar .  300-4,500 

Communications  Equipment  MTBF  (Hours) 

Ground  Radio .  5,000-20,000 

Portable  Ground  Radio .  1,000-3,000 

Airborne  Radio .  500-10,000 

Ground  Jammer .  500-2,000 

Computer  Equipment  MTBF  (Hours) 

Ground  Computer .  1,000-5,000 

Ground  Monochrome  Display .  1 5,000-25,000 

Ground  Color  Display .  2,500-7,500 

Ground  Hard  Disk  Drive .  5,000-20,000 

Ground  Tape  Storage  Unit .  2,500-5,000 

Ground  Printer .  2,000-8,000 

Ground  Modem .  20,000-50,000 

Miscellaneous  Equipment  MTBF  (Hours) 

Airborne  Countermeasures  System .  50-300 

Airborne  Power  Supply .  2,000-20,000 

Ground  Power  Supply .  10,000-50,000 
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Qmieral  Process 


CoovMsioool 
U*ar  Needs  to 
Translatable 
Operational 
Terms 


detailed  Process 


Translation  of 

Operational  Terms 

to  Contractual 

Terms 

(Potential 

C^tractual 

Requiremertls) 


Check  Requirements 
for  Realism 


Contractual 

Requirements 


Figure  R1-1 :  Quantitative  Reiiabiiity  Requirement  Deveiopment  Proceaa 
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Figur*R1>1  Not*s: 

1.  User  Needs  Cases 


CaM 

LogisticBRaUablllty 

MiulonRallablllty 

Comments 

1 

Specified 

Specified 

2 

Specified 

Not  specified 

Delete  steps  D.  H,  1 

3 

Not  specified 

Specified 

4 

Not  specified 

Not  specified 

Delete  steps  D.  H,  1 

2.  A 10-20%  reliability  improvement  factor  is  reasonable  for  advancement 
of  technology. 

3.  Adjustment  of  data  to  use  environment  may  be  required  (see  Topic  A1 1 ).  See 
Appendix  8  for  R&M  data  sources. 

4.  Reliability  requirements  necessitating  redundancy  add  weight,  cost  and  power. 

5.  Alternate  forms  of  user  requirements  should  be  converted  to  MTBM's  to  enable 
translation. 
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Scope  of  Requirements: 

Unique  maintainability  parameters  need  to  be  specified  for  three  basic  levels 

of  repair: 

•  Organizational  Level — Repair  at  the  system  location.  Usually  involves 
replacing  plug-in  modules  and  other  items  with  relatively  short  isolation  and 
replacement  times. 

•  Intermediate  Level — Repair  at  an  intermediate  shop  facility  which  has  more 
extensive  capabilities  to  repair  lower  hardware  indenture  levels. 

•  Depot  Level — Highly  specialized  repair  facility  capable  of  making  repairs  at  all 
hardware  indenture  levels.  Sometimes  the  original  equipment  manufacturer. 

Recent  Air  Force  policy  has  promoted  the  concept  of  two  level  maintenance  in 

place  of  the  traditional  three  level  system.  Under  this  concept  the  classification  is; 

■  On-equipment — Maintenance  actions  accomplished  on  complete  end  items. 

•  Off-equipment — In-shop  maintenance  actions  performed  on  removed 
components. 

Parameters  which  need  to  be  specified  vary  with  the  level  of  repair  being 

considered.  Key  maintainability  parameters  include; 

•  Mean  time  to  repair  (MTTR)— Average  time  required  to  bring  system  from  a 
failed  state  to  an  operational  state.  Strictly  design  dependent.  Assumes 
maintenance  personnel  and  spares  are  on  hand  (i.e.,  does  not  include  logistics 
delay  time).  MTTR  is  used  interchangeably  with  mean  corrective  maintenance 
time  (Met). 

■  Mean  maintenance  manhours  (M-MMH) — Total  manpower  per  year 
(expressed  in  manhours)  required  to  keep  the  system  operating  (not  including 
logistics  delay  time). 

•  Mean  lime  to  restore  system  (MTTRS)— The  average  time  it  takes  to  restore 
a  system  from  a  failed  state  to  an  operable  state,  including  logistics  delay  time 
(MTTRS  =  logistics  delay  time  +  MTTR).  Logistics  delay  time  includes  all  time 
to  obtain  spares  and  personnel  to  start  the  repair. 

•  Preventive  maintenance  (PM) — Preventive  maintenance.  Time  associated 
with  the  performance  of  all  required  preventive  maintenance.  Usually  expressed 
in  terms  of  hours  per  year. 


Operational  Constraints: 

Basic  maintainability  requirements  are  determined  through  an  analysis  of  user 
operational  constraints.  Operational  constraints  include. 

•  Operating  hours  per  unit  calendar  time  and/or  per  mission. 

•  Downtime,  maintenance  time,  or  availability  constraints. 
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REQUIREMENTS— TOPIC  R2 


•  Mobility  requirements. 

•  Attended/unattended  operation. 

•  Self-sufficiency  constraints. 

•  Reaction  time. 

•  Operational  environment. 

•  Skill  levels  of  maintenance  personnel. 

•  Manning. 

•  Types  of  diagnostics  and  maintenance  support  equipnnent  which  can  be  made 
available  or  implemented  (Built-in-test,  manual  test  equipment,  external 
automatic  test  equipment,  etc.). 

•  Levels  at  which  repair  takes  place. 

•  Use  of  off-the-shelf  equipment  versus  newly  designed  equipment. 

How  to  Develop  Requirements: 

The  best  guidance  available  is  to  provide  a  range  of  typical  values  usually  applied 
for  each  parameter: 
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Table  R2-1 :  Typical  Maintainability  Values 


Organizationai 

Intarmedtate 

Depot 

MTTR 

.5- 1.5  hr 

.5-3  hr 

1-4  hr 

M-MMH 

Note  1 

Note  1 

Note  1 

MTTRS 

1  -8  hrs  (Note  2) 

NA 

NA 

PM 

2-15  hr/yr 

NA 

NA 

Note*: 


1 .  M-MMH  depends  on  the  number  of  repair  visits  to  be  made,  the  MTTR  lor  each  repair  visit  and  the  number  of 
maintenance  personnel  required  for  each  visit.  Typical  calculations  of  the  mean  maintenance  manhours  per  year 
irvtlude; 

a.  Immediate  rrwintenance  of  a  continuously  operated  system:  M-MMH  =  (8760  hr/yr)/(MTBF)  x  (MTTR)  x 
(maintenance  personnel  per  repair)  +  (PM  hours  per  year)  (Maintenance  personnel). 

b  Delayed  maintenance  of  a  fault  tolerant  system:  M-MMH  =  (rtumber  of  expected  repair  visits)  «  (time  lor  each 
visit)  X  (maintenance  personnel  per  visit)  +  (PM  hours  per  year)  (maintenance  personnel) 

c  Maintenance  of  a  continuously  operated  redundant  system  allowed  to  operate  until  failure.  M-MMH  =  (8760  hr 
yr)/(MTBCF)  x  (time  for  each  visit)  x  (maintenarx»  personnel  per  visit)  -r  (PM  hours  per  year)  (Maintenance 
personnel). 

T ime  tor  each  visit  is  the  number  of  repairs  to  be  made  times  the  MTTR  lor  each  repair  if  repairs  are  made  in  series 
2  For  unique  systems  that  are  highly  redundant.  MTTRS  may  be  specified  as  the  switch  time 
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ility  Requirements 


Scope  of  Roquiromonts: 

Parameters  that  need  to  be  specified  for  each  repair  level: 

•  Fault  Detection — A  process  which  discovers  the  existence  of  faults. 


•  Fault  Isolation — Where  a  fault  is  known  to  exist,  a  process  which  ideniifies  the 
location  of  that  fault. 


•  False  Alarms — An  indication  of  a  fault  where  no  fault  exists  such  as  operator 
error,  transient  condition,  BIT  design  deficiency. 


Parameters  are  sometimes  expressed  in  the  form  of  rates  or  fractions  such  as. 


•  Fraction  of  Faults  Detected  (FFD) — The  quantity  of  faults  detected  by  on¬ 
board  test  divided  by  the  quantity  of  all  faults  detected  by  all  means 
(including  manual). 

•  Fraction  of  Faults  Isolated  (FFI) — The  fraction  of  on-board  test  detected  faults 
correctly  isolated  to  the  replacable  unit. 


•  False  Alarm  Rate  (FAR) — The  frequency  of  occurrence  of  false  alarms. 


Scope  of  Diagnostics: 

•  integrated — Use  of  built-in-test  (BIT)  which  operates  on  demand  or 
automatically. 

•  External — Special  purpose  test  equipment  that  must  be  connected  by  a 
maintenance  technician. 

•  Manual — Testing  that  requires  the  use  of  technical  manuals,  troubleshooting 
procedures  and  general  purpose  test  equipment  (e  g.,  voltmeter)  by  a 
maintenance  technician. 


How  to  Develop  Requirements: 

The  best  guidance  available  is  to  provide  a  range  of  typical  values  usually  applied 
for  each  parameter. 
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Table  RS-I :  Typical  Testability  Values 


Fault  Detection  (all  means) 


Fault  Detection  (BIT) 


Fault  Isolation 
Eight  or  less  modules 
Three  or  less  modules 
One  module 
False  Alarms 


%  Capability 

Repair  Level 

90-100 

Organizational 

lOO 

Intermediate 

100 

Depot 

90-98 

Organizational 

95-98 

Intermediate 

95-100 

Depot 

95-100 

All 

90-95 

All 

80-90 

All 

1000-5000  hours  between  alarm 


Note:  Intormation  and  guidance  concerning  testability  diagnostics  specilication.  design,  assessment,  and  evaluation 
IS  currently  being  developed  and  organized  under  RADC  contract  F30602-87-C-0099  The  documentation 
forthcoming  is  in  the  form  ol  a  Government  Program  Managers  Guide.  Contractor  Program  Managers  Guide 
arvj  a  Testability  Design  Encyclopedia  Release  of  the  documents  is  expected  by  January  1989  Information 
about  obtaining  these  documents  can  be  obtained  from  RADC  RBET  at  (315)  330-4726 
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hardware  is  not  intended  lor  operational  use. 

Likely  to  Go  Directly  to  Production  •  Advanced  development  hardware  is  being  •  Significant  R&M  requirements  are  necessary 

developed  for  production  prototype  purposes.  (e  g-.  RAM  prediction,  part  derating,  parts  control, 

FRACAS). 


Full  Scale  Development  .  Develop  and  lest  a  prototype  model  for  •  Extensive  R&M  requirements  are  considered 

- production  purposes _ _ _ _  necessary  (e  g. ,  all  the  above  plus  testing). 

Production  •  To  build,  lest  and  accept  operational  units  without  •  Emphasis  is  on  quality  assurance  and  ESS  tasks. 

degrading  the  capabilities  previously  designed 
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Topic  R5:  Reliability  and  Maintainability  Task  Application/Priorities 


REQUIREMENTS— TOPIC  R5 
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Effects  of  Storage.  Handling,  efc  |  |RRRjOOO|  Inchn  DI-R-7095 


Test  ft  Evaluation  Tasks 


REQUIREMENTS— TOPIC  R5 
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REQUIREMENTS— TOPIC  R6 


In  order  for  the  government  to  receive  outputs  from  the  required  contractor 
performed  tasks,  the  appropriate  deliverables  must  be  specified  in  the  Contract 
Data  Requirements  List  (CDRL).  The  content  of  these  CDRL  items  is  specified  by 
reference  to  standard  Data  Item  Descriptions.  The  timing  and  frequency  of  the 
required  reports  must  be  specified  in  the  CDRL. 


Table  R6-1 :  Data  Items  &  Delivery  Dates 


Title 

Recommended  Delivery  Date 

Reliability 

DI-R-7040 

Bum-in  Test  Report 

60  days  after  end  of  testing 

DI-R-7041 

Failure  Summary  &  Analysis  Report 

Start  of  testing,  monthly 

DI-R-7079 

Reliability  Program  Plan 

90  days  prior  to  PDR 

DI-R-7080 

Reliability  Status  Report 

90  days  prior  to  PDR  &  bimonthly 

DI-R-7083 

Sneak  Circuit  Analysis  Report 

30  days  prior  to  PDR  &  CDR 

DI-R-7085 

FMECA  Report 

30  days  prior  to  CDR 

DI-R-7086 

FMECA  Plan 

90  days  prior  to  PDR 

DI-R-7094 

Reliability  Block  Diagram  &  Math  Model  Report 
(Note  4) 

30  days  prior  to  PDR  &  CDR 

DI-R-7095 

Rel  Pred  &  Documentation  of  Supporting  Data 
(Note  3) 

30  days  prior  to  PDR  &  CDR 

DI-R-7100 

Rel  Report  for  Exploratory  Development  Models 

30  days  prior  to  end  of  contract 

DI-R-35011 

Critical  Item  Control  Plan 

30  days  prior  to  PDR 

DI-MISC-80071 

Pan  Approval  Request  (Note  2) 

As  Required 

DI-RELI-80247 

Thermal  Survey  Report 

30  days  prior  to  PDR  &  after 
testing 

DI-RELI-80248 

Vibration  Survey  Report 

30  days  prior  to  PDR  &  after 
testing 

DI-RELI-80250 

Reliability  Test  Plan 

90  days  prior  to  start  of  testing 

DI-RELI-80251 

Reliability  Test  Procedures 

30  days  prior  to  start  of  testing 

DI-RELI-80252 

Reliability  Test  &  Demo  Report 

60  days  after  end  of  testing 

DI-RELI-80253 

Failed  Item  Analysis 

As  Required 

DI-RELI-80254 

Corrective  Action  Plan 

30  days  after  end  of  testing 
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Title 

Recommended  Delivery  Date 

Maintainability 

DI-R-7103 

Maintainability  Program  Plan 

90  days  prior  to  PDR 

DI-R-7104 

Maintainability  Status  Report 

90  days  prior  to  PDR  &  Bimonthly 

DI-R-7105 

Data  Collection,  Analysis  &  Corrective  Action  Rpt 

As  Required 

DI-R-7106 

Maintainability  Modeling  Report 

30  days  prior  to  PDR  &  COR 

DI-R-7107 

Maintainability  Allocations  Report 

30  days  prior  to  PDR  &  COR 

DI-R-7108 

Maintainability  Predictions  Report 

30  days  prior  to  PDR  &  COR 

Dl-R  7109 

Maintainability  Analysis  Report 

30  days  prior  to  PDR  &  CDR 

DI-R-7110 

MeiPteinat'Iity  Design  Crilena  Plan 

30  days  piior  to  PDR 

DI-R-7111 

Inputs  to  the  Detailed  Maintenance  Plan  & 
Logistics  Support 

DI-R-7112 

Maintainability  Demo  Test 

90  days  prior  to  start  of  testing 

DI-R-7113 

Maintainability  Demo  Report 

90  days  prior  to  start  ol  testing 

Testability 

DI-R-7080  & 
7105 

(See  Reliability  Data  Mem  List) 

DI-R-7112& 

7113 

(See  Maintainability  Data  Item  List) 

DI-T-7198 

Testability  Program  Plan 

90  days  prior  to  PDR 

DI-T-7199 

Testability  Analysis  Report 

30  days  prior  to  PDR  &  CDR 

Notes: 

1  Combine  RAM  data  items  into  a  single  submission  whenever  possible 

2  OESC  must  be  included  on  the  distribution  list  of  this  CDRL  Data  Item  The  government  lead  system  engineer  must 
establish  contact  and  agreement  for  review  with  DESC  and  DISC  Contacts  are  ( 1 )  Electronics.  DESC  DESC' 
ERA.  AV:  986-5431  or  (513)296-5431 ,  and  (2)  Mechanical  Parts.  DISC:  DISC/ESM.  Autovon  442-4395  or  (215) 
697-4395 

3  Add  the  following  to  the  data  item  if  a  thermal  analysis  will  be  required;  "The  junction  and  hot-spot  temperatures 
listed  in  the  Reliability  Predictions  Report  shall  be  obtained  from  a  detailed  thermal  analysis,  conducted  down  to  the 
pan  level  The  repon  shall  irxilude:  (1)  adescnption  of  the  equipment  analyzed.  (2)  sources  of  estimates  of  pan 
dissipations.  (3)  thermal  resistances  (values  used,  assumptions,  coolant  flow  rates).-  (4)  sources  of  estimates  ot 
sink  temperatures,  and  (5)  analysis  method  " 

4  Add  to  the  data  item :  Reliability  allocations  shall  be  pedormed  lor  both  mission  and  senes  reliability  and 
documented  to  the  circuit  card  level 

5  POR  is  assumed  to  take  place  about  9  months  after  the  contract  award  date 
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rbnfor^ 


Proposal  preparation  guidance  should  be  provided  in  the  request  for  pro|X)sal 
(RFP)  package  to  guide  the  contractor  in  providing  the  information  most  needed  to 
properly  evaluate  the  R&M  area  during  source  selection.  This  is  part  of  the 
requirements  definition  process. 

Depending  on  the  sco(>e  of  the  R&M  requirements  specified,  information  such  as 
the  following  should  be  requested  for  inclusion  in  the  proposal; 


•  Preliminary  R&M  analysis/models  and  estimates  of  values  to  be  achieved  (to  at 
least  the  line  replaceable  unit  (LRU)  level). 

•  Design  approach  (including  iheunai  dt^siyn,  pans  derating,  and  parts  control). 

•  R&M  organization  and  its  role  in  the  overall  program. 

•  Key  R&M  personnel  experience. 

•  Schedules  for  all  R&M  tasks. 

•  Description  of  R&M  design  guidelines  criteria  to  be  used  and  trade  studies  and 
testing  to  be  performed. 


(Note:  It  is  critical  that  qualified  R&M  personnel  take  part  in  the  actual  evaluation 
of  technical  proposals.  The  R&M  engineer  should  make  sure  this  happens  by 
agreement  with  program  management.) 
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.5-.  -.SJaR**  XHBf*-:? 

st€stimati^ 


Independent  Variable 
Range 

Reliability  Task 

Indapandant  Variable  Manhour  Model 

Min 

Max 

R4M  Program  Plan 

Number  of  MIL-STD-785/470  tasks 

required  (NOT)  2.73  (NOT)' 

4 

22 

FRACAS 

Duration  of  FRACAS  implementation 
in  months  (DOI)  8.25  (DO/)' 

2.5 

38 

Reliability  Modeling/ 
Allocation 

(1)  Modeling  and  Allocation 
Complexity 

4.05  (MAC)' (NOU) 

MAC 

1  Series  System 

2  Simple  Redundancy 

3  Very  Complex  Redundancy 

1 

3 

(2)  Number  of  Items  in  Allocation 
Process  (NOU) 

7 

445 

Reliability  Prediction 

(1)  Level  of  Detail 

4.54(LOD)'(RF)' 

(POC) 

LOO 

1  Prediction  Exists 

2  Prediction  made  using  similar 
system  data 

3  Full  MIL-HDBK-217  Stress 
Prediction 

1 

3 

(2)  Report  Formality 

RF 

1  Interna!  Report 

2  Formal  Report  Required 

1 

2 

(3)  Percent  Commercial  Hardware 
Used 

POC  Percent 

4  0-25 

3  26-50 

2  51-75 

1  76-100 

1 

4 

FMEA 

Number  of  unique  items  requiring 
FMEA  (NOI) 

NOI  =  Number  of  equipment  for 
equipment  level  FMEA 

NOI  =  Number  of  circuit  cards  for 
piece  part  and  circuit  level  FMEA  s 

17.79  (NOI) 

3 

206 
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REQUIREMENTS— TOPIC  R8 


■nd^pcndMit  VarlabI* 
Rang* 


RellaMUtyliMk 

Independent  Varfabla 

Manhour  Model  Min 

Max 

Reliability  Testing 

Hardware  Complexity 

182.07  (HC) 

HC  Part  Count 

1  <15000 

2  15000-25000 

3  >25000 

1 

3 

R&M  Management 

All  of  the  above 

16  percent  of  above 
subtotal 

Parts  Program 

Number  of  nonstandard  parts 

(No.  of  parts) 

(40  hr/part) 

Exampla:  A  program  requires  the  following  tasks  The  manhour  estimate  is  calculated  as  follows: 


1  36  month  FRACAS  program 

2  Reliability  modeling  (no  redundancy)  and  allocation  for  445  circuit  cards 

3  Stress  type  reliability  prediction  (formal  report.  50%  commercial  hardware) 

4  Equipment  level  FMEA  (3  major  assemblies) 


Manhours 

8.25(36)* 

10.692 

4.05(1  )*(445) 

1.802 

4  54(3)*(2)*(3) 

490 

17  79(3) 

53 

•  13.037 

R&M 

Management 

16% 

2.086 

Total  Manhours  15.123 

Note;  For  further  detailed  information  on  cost  model  development,  see  RADC-TR-87-50.  R&M  Program  Cost  Dnvers 
AD  number  A182773 
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Section  S 


What’s  my  role  in  the  SOURCE  SELECTION  process  and  what  criteria 

should  be  used  to  evaluate  contractor  proposals? 

R&M  Engineer’s  Role: 


R1-R7  Establish  Clear  Requirements .  11—30 

R7  Specify  Information  for  Proposals .  30 

SI  Evaluate  Proposals  for  R&M .  35 
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For  More  Information 

AFR  70-15  "Source  Selection  Policy  and  Procedures” 
AFR  70-30  "Streamlined  Source  Selection  Procedures" 
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ilityShd 


•  llRderstanding 

1 .  Does  the  contractor  show  understanding  of  the  importance  of  designing  in 
R&M&T  in  the  effort? 

2.  Does  the  contractor  show  a  firm  understanding  of  R&M&T  techniqui^s, 
methodology,  and  concepts? 

3.  Does  the  contractor  indicate  understanding  of  the  role  of  testability/ 
diagnostics  on  maintainability  and  maintenance? 

4.  Does  the  contractor  understand  integrated  diagnostics  design  principles? 

5.  Does  the  contractor  note  similar  successful  R&M&T  efforts? 


•  Approach 

Management 

1 .  Is  an  R&M&T  manager  identified,  and  are  his/her  experience  and 
qualifications  adequate  in  light  of  the  scope  of  the  overall  program? 

2.  Are  the  number  and  experience  of  R&M&T  personnel  assigned  to  the 
program,  and  the  number  of  manhours  adequate,  judged  in  accordance  with 
the  scope  of  the  overall  program? 

3.  Does  the  R&M&T  group  have  adequate  stature  and  authority  in  the 
organizational  framework  of  the  program  (e.g.,  they  should  not  fall  under  direct 
control  of  the  design  group)? 

4.  Does  the  R&M&T  group  have  an  effective  means  of  crosstalk  and  feedback  of 
information  between  design  engineers  and  higher  management? 

5.  Does  the  R&M&T  manager  have  adequate  control  over  R&M&T  for 
subcontractors  and  vendors? 

6.  Is  the  testability  diagnostics  function  integrated  into  the  R&M  program? 

Design 

1 .  Are  design  standards,  guidelines  and  criteria  such  as  part  derating,  thermal 
design,  modular  construction,  ESS,  and  testability  cited? 

2.  Is  the  contractor’s  failure  reporting  and  corrective  action  system  (FRACAS)  a 
closed  loop  controlled  process? 

3.  Is  there  a  commitment  to  the  required  parts  control  program  (e  g.,  MIL-M- 
38510,  MIL-STD-883B,  etc.)?  Are  approval  procedures  described/proposed 
for  nonstandard  parts? 

4.  Are  system  design  reviews  (internal  and  external)  required  regularly? 
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5.  Are  tradeoff  studies  proposed  for  critical  design  areas? 

6.  Is  a  time-phasing  of  R&M&T  tasks  provided  along  with  key  program  mile¬ 
stones? 

7.  Are  areas  of  R&M&T  risk  identified  and  discussed? 

8.  Does  tne  contractor  include  consideration  of  software  reliability? 

9.  Does  the  contractor  describe  his  plan  for  testability/diagnostics  design  and  the 
potential  impacts  on  reliability  and  maintainability? 

1 0.  Does  the  contractor  identify  tools  to  be  used  to  generate  lest  vectors  and  other 
diagnostic  procedures  for  BIT  and  ATE  (automatic  test  equipment)? 

Analysis/Test 

1 .  Are  methods  of  analysis  and  math  models  presented? 

2.  Are  the  R&M&T  prediction  and  allocation  procedures  described? 

3.  Has  the  time  phasing  of  the  R&M&T  testing  been  discussed,  and  is  it 
consistent  with  the  overall  program  schedule? 

4.  Is  adequate  time  available  for  the  test  type  required  (such  as  maximum  time 
for  sequential  test). 

5.  Is  the  ESS  program  consistent  with  the  requirements  in  terms  of  meti  odology 
and  scheduling? 

6.  Does  the  contractor  make  a  commitment  (o  predict  the  design  requirf-ment 
MTBF  prior  to  the  start  of  testing? 

7.  Are  the  resources  (test  chambers,  special  equipment,  etc.)  needed  to  perform 
all  required  testing  identified  and,  is  a  commitment  made  to  their  availability? 

•  Compliance 

Design 

1 .  Does  the  contractor  indicate  compliance  with  all  required  military 
specifications  for  reliability,  maintainability  and  testability? 

2.  Is  adequate  justification  (models,  preliminary  estimates,  data  sources,  etc.) 
provided  to  backup  the  claims  of  meeting  R&M&T  requirements? 

3.  Is  there  an  explicit  commitment  to  meet  any  ease  of  maintenance  and 
preventive  maintenance  requirements? 

4.  Is  there  an  explicit  commitment  to  meet  the  built-in-test  (BIT)/fault-isolation- 
test  (FIT)  requirements  (FFD,  FFI  and  FAR)? 

5.  Is  each  equipment  environmental  limitation  specified  and  do  these  conditions 
satisfy  the  system  requirements? 

6.  Are  all  removable  modules  keyed? 
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7.  Will  derating  requirements  be  adhered  to  and  are  methods  of  verifying 
derating  requirements  discussed? 

Analysis/Test 

1 .  Is  a  commitment  made  to  perform  a  detailed  thermal  analysis? 

2.  Will  the  contractor  comply  with  all  R&M&T  required  analyses? 

3.  Is  there  an  explicit  commitment  to  perform  all  required  environmental 
stress  screening? 

4.  Does  the  contractor  comply  with  all  system  level  R&M&T  test  requirements? 
Will  the  contractor  demonstrate  the  R&M&T  figures  of  merit  (MTBR  MTTR, 
FFD,  FFI  and  FAR)  using  the  specified  accept/reject  criteria? 

5.  Does  the  contractor  comply  with  the  specification  (or  other  commonly 
specified)  failure  definitions? 

6.  Does  the  contractor  agree  to  perform  thermal  verification  tests  and  derating 
verification  tests? 

Data 

1 .  Is  there  an  explicit  commitment  to  deliver  and  comply  with  all  of  the  required 
R&M&T  data  items? 

CAUTION:  Procuring  activities  have  differing  rules  for  the  format  of  source 
selection  criteria.  The  above  issues  should  be  addressed  regardless  of 
particular  format. 
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Section  D 


What  is  my  role  in  developing  design  requirements  and  in  the  DESIGN 

PROCESS  of  a  development  program? 

R&M  Engineer’s  Role: 


Select  Design  Requirements 

D1  Part  Stress  Derating .  41 

D2  Thermal  Design  Limitations .  46 

D3  Parts  Control .  48 

Evaluate  Design 

D4  Review  Questions .  53 

D5  Reliability  Critical  Items .  59 

Appendix  4  Example  Design  Guidelines .  A>29 
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For  More  Information 


MIL-STD-883 

MIL-STD-965 

MIL-STD-1521 

MIL-HDBK-251 

MIL-HDBK-338 

MIL-M-38510 

MIL-S-19500 


“Test  Methods  and  Procedures  for  Microelectronics" 
“Parts  Control  Program" 

“Technical  Reviews  and  Audits  for  Systems,  Equipments, 
and  Computer  Software” 

“Reliability/Design  Thermal  Applications” 

“Electronic  Reliability  Design  Handbook” 

“Microcircuits,  General  Specification  for" 

“Semiconductor  Devices,  General  Specification  for" 


AFSC  Pamphlet  800-27  “Parts  Derating  Guidelines" 

R  ADC-TR-82-172  “RADC  Thermal  Guide  for  Reliability  Engineers" 
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ppic  I 


The  practice  of  limiting  electrical,  thermal  and  mechanical  stresses  on  parts  to 
levels  below  their  specified  ratings  is  called  derating.  If  a  system  is  expected  to  be 
reliable,  one  of  the  major  contributing  factors  must  be  a  conservative  design 
approach  incorporating  realistic  derating  of  parts.  Table  D1  -1  defines  the  key 
factors  for  determining  the  appropriate  level  of  derating  for  the  given  system 
constraints.  Table  D1  -2  indicates  the  specific  derating  factors  for  each  part  type. 


Table  D1-1 :  Part  Derating  Level  Determination 


Factors  Score 


Reliability 

Challenge 


System  Repair 


Salety 


Size.  Weigh; 


Lite  Cycle 


For  proven  design,  achievable  with  standard  parts/circuits 

For  high  reliability  requirements,  special  design  features  needed 

For  new  design  challenging  the  state-ol-the-art,  new  corxtepf 

For  easily  accessible,  quickly  and  economically  repaired  systems 

For  high  repair  cost,  limited  access,  high  skill  levels  required,  very  low 
downtimes  allowable 

For  nonaccessible  repair,  or  economically  unjustifiable  repairs 

For  routine  safety  program,  no  expected  problems 

For  potential  system  or  equipment  high  cost  damage 

For  potential  jeopardization  of  life  of  personnel 

For  no  significant  design  limitation,  standard  practices 

For  special  design  features  needed,  difficult  requirements 

For  new  concepts  needed,  severe  design  limitations 

For  economical  repairs,  no  unusual  spare  part  costs  expected 

For  potentially  high  repair  cost  or  unique  high  cost  spares 

For  systems  that  may  require  complete  substitution 


1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 


Instructions:  Select  score  for  each  factor,  sum  and  determine  derating  level  or  parameter 


Derating  Level 


Total  Score 


I  11-15 

II  7-10 


III 


6  or  less 
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Rotating  SAW 

Bearing  Load  TS^'o  90%  90%  Input  Power  (dB) 

Operating  Temp  (°C)  40  25  15  (Freq  >  500  MHz) 

(L-%  Rated)  Input  Power  (dB) 

(T  from  Max  Limit)  (Freq  <  500  MHz) 


DESIGN— TOPIC  D1 


iO 

CNJ 


tn 

CNJ 


ie> 

CVI 


moo 

^  in 


moo 
^  m 


> 

z 

CD 


S 


m  o 
eo 


Q. 

«3 

^  Jr 

C  C  O 
^  Q}  Cv 

t  fc  C 
3^0 

O  O  O 


</) 


RADC  RELIABILITY  ENGINEER  S  TOOLKIT 


45 


DESIGN— TOPIC  D2 


tions 


One  of  the  important  variables  in  system  reliability  is  temperature.  Therefoie,  the 
thermal  design  of  a  system  must  be  planned  and  evaluated.  Full  discussion  of  this 
topic  is  beyond  the  scope  of  this  document  but  it  is  important  to  point  out  to  a 
reliability  engineer  what  limitations  there  are  for  common  thermal  design 
approaches.  Tables  D2-1  and  D2-2  summarize  the  most  common  cooling 
techniques  for  electronics  and  their  limitations. 


Table  D2-1 :  Cooling  Techniques  Limitations  (Per  Unit  Volume) 


OiMipation  Per  Unit  VOtum*  (W/ft>) 

Sufficiant  Cooling  Tachniqua  (Most  Appllcatlona) 

0-300* 

Free  convection  to  ambient  air 

300*-1000 

Forced  air  convection 

>1000 

Custom  design,  thermal  considerations  should  have  tc  ’  priority 
in  physical  design 

‘lOOW  If’  instead  of  300W'tt^  if  box  is  poorly  ventilated  and  thermally  sensitive  parts  are  mounted  honzooi  illy 
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Table  D2>2:  Cooling  Techniques  Limitations  (Per  Unit  Area) 


Cooling  Tochnique 


Impingement 
— Free  Convection 
— Forced  Air 


Coldwall 


Maximum  Cooling  Capacity 
W/M»  _ W/IN»  Dascription 


800 

3000 


1500 


0.5 

2 


1 


Example;  A  9"  x  5"  printed  circuit  board  using  free  convection  cooling  would  be 
limited  to  about  22.5  watts. 
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Managing  a  parts  control  program  is  a  highly  specialized  activity  and  does  not 
typically  fall  under  the  system's  R&M  engineer's  responsibility.  Howrever,  because 
of  the  interrelationship  of  parts  control  and  good  system  reliability,  it  is  important 
that  R&M  engineers  and  program  managers  have  a  general  understanding  of  the 
parts  control  discipline.  Parts  control  questions  which  are  often  asked  include: 

•  Why  do  parts  control? 

•  What  are  the  various  “tools"  to  accomplish  parts  control? 

•  What  is  a  JAN  part,  a  MIL-STD-883  part,  a  standard  military  drawing  (SMD)  part, 
and  a  verKlor  equivalent  part? 

Why  do  parts  control?  Since  the  invention  of  semiconductors,  users  could  never 
be  sure  that  a  device  purchased  from  one  manufacturer  would  be  an  exact 
replacement  for  the  same  device  obtained  from  another  supplier.  Major  differences 
in  device  processing  and  electrical  testing  existed  among  suppliers.  Because  of 
the  importance  of  semiconductors  to  military  programs,  the  government  introduced 
standard  methods  of  testing  and  screening  devices  in  1968.  Devices  which  were 
tested  and  screened  to  these  methods  were  then  placed  on  a  government 
approval  list  called  the  qualified  parts  list  (OPL).  Through  this  screening  and 
testing  process,  a  part  with  known  quality  and  performance  characteristics 
is  produced. 

What  are  the  various  “tools”  to  accomplish  parts  control?  The  govei  nment 
has  subdivided  parts  into  three  basic  classifications:  (1 )  microelectronics,  (2) 
semiconductors  (e.g.,  transistors,  diodes,  etc.),  and  (3)  electrical  parts  (e  g., 
switches,  connectors,  capacitors,  resistors,  etc.).  For  each  class,  part  specification 
and  test  method  documents  have  been  developed.  Table  D3-1  summarizes  key 
documents  and  their  content. 


What  is  a  JAN  part,  a  MIL’STD-883  part,  a  standard  military  drawing  (SMD) 
part,  and  a  vendor  equivalent  part?  The  primary  difference  in  these  descriptions 
is  that  each  of  these  part  classes  has  undergone  different  levels  of  screening  and 
certification.  Certification  involves  specifying  and  documenting  the  part 
manufacturing  process.  It  also  involves  government  and  manufacturer  agreement 
on  a  detailed  part  specification.  This  ensures  consistent  part  quality  and  known 
performance.  Table  D3-2  summarizes  common  classes  of  parts  and  what  these 
classifications  signify.  Table  D3-3  summarizes  MIL-STD-883C  screening 
procedures  and  is  included  to  give  the  reader  a  feel  for  the  wide  range  of  tests 
required.  Topic  At  1  shows  the  impact  of  the  various  part  quality  designations  on 
system  reliability. 
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Table  D3-1 :  Key  Parts  Control  Documents  and  Their  Content 

Document  Title  Content 


MIL-M'38510  General  Specification  Provides  detailed  specification  requirements  in  the  form  of 
for  Microcircuits  "slash  sheets"  for  several  hundred  of  the  most  commonly 

used  microcircuits.  Covers  screening  requirements 
(referenced  to  MIL-STD-883).  electrical  testing,  quality 
conformance,  physical  dimensions,  configuration  control  for 
critical  manufacturing  processing  steps  and  production  line 
certification 


MIL-STO-883  Test  Methods  and 
Procedures  for 
Microelectronics 


Provides  uniform  methods  and  procedures  for  testing 
microelectronic  devices.  Structured  into  five  classes  of  test 
methods:  1000  Class  addresses  environmental  tests.  2000 
Class  addresses  mechanical  tests,  3000  Class  addresses 
electrical  tests  for  digital  circuits,  4000  Class  addresses 
electneal  tests  for  linear  circuits,  and  5000  Class  addresses 
test  procedures.  The  tests  covered  include  moisture 
resistance,  seal  test,  neutron  irradiation,  shock  and 
acceleration  tests,  dimensional  tests,  input/output  current 
tests,  and  screening  test  procedures  to  name  a  few  Two  test 
levels  are  described;  Class  B  and  Class  S  Class  S  is  geared 
toward  space  qualified  parts  and  requires  a  host  ol  tests  not 
performed  on  Class  B  parts  (e.g.,  wafer  lot  acceptance,  bond 
pull,  particle  impact  noise  detection,  serialization,  etc.). 


MIL-S-19500  General  Specification  Provides  detailed  specification  sheets  establishing  general 
for  Semiconductors  and  specific  requirements  including  electrical  characteristics. 

mechanical  characteristics,  qualification  requirements, 
inspection  procedures  and  test  methods. 


MlL  STD-750  Test  Methods  for  Provides  uniform  methods  and  procedures  for  testing 

Semiconductor  Devices  semiconductors.  Structured  into  five  classes  of  lest  methods 
1000  Class  addresses  environmental  tests.  2000  Class 
addresses  mechanical  characteristics.  3000  Class  addresses 
electrical  characteristics  tor  transistors.  3100  Class  addresses 
circuit  performance  and  thermal  resistance  measurements, 
and  the  3200  Class  addresses  low  frequency  tests. 


Provides  uniform  methods  for  testing  electronic  and  electrical 
parts  Structured  into  three  classes  of  test  methods:  100 
Class  addresses  environmental  tests.  200  Class  addresses 
physical  characteristic  tests,  and  300  Class  addresses 
electrical  characteristic  tests.  These  tests  are  not  tied  to  a 
single  part  specification  document  as  with  microelectronics 
and  semiconductors,  but  rather,  numerous  specifications  for 
various  component  types 

MIL  STD-965  Parts  Control  Program  Provides  parts  control  procedures  to  be  used  in  the  design 

and  development  ol  military  equipment,  including  the 
submission,  review  and  approval  of  a  Program  Parts 
Selection  List.  Generally,  an  overall  guide  for  the 
implementation  and  management  of  a  parts  control  program 
The  document  provides  for  two  basic  management 
procedures  Procedure  I  is  applicable  to  a  majority  of 
programs  and  does  not  make  use  of  a  formal  parts  control 
board  Procedure  II  requires  a  formal  parts  control  board  and 
is  recommended  for  consideration  where  there  is  an 
aggregation  of  contractor/subcontractors 


M I L-  STD-202  Test  Methods  for 

Electronic  and  Electrical 
Parts 
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Table  D3-2:  Microelectronics  Classifications  and  Descriptions 


Part  Claaaificatlon  Part  Claaalfication  Dascription 


JAN  or  MIL-M-38510  Parts 


MIL-STD-883orSMD 
(Standard  Military  Drawing) 
Parts 


Vendor  Equivalent  Parts 


These  parts  have  a  detailed  specification  (slash  sheet)  in  MIL-M- 
38510  which  controls  all  rrtechanical.  electrical,  and  functional 
parameters  of  the  part.  Additionally,  the  manufacturing  process  flow 
is  government  certified,  the  devices  are  screened  to  MIL  STO'B83 
requirements,  and  the  devices  are  subjected  to  rigorous  quality 
conformance  testing 

"SMD"  and  "883"  are  used  interchangeably  to  describe  part  quality 
A  part  which  is  marked  "MIL-STD-883"  irxticates  that  the  part  has 
bMn  tested  to  a  host  of  MIL-STD-883  requirements.  These  include 
screening  requirements,  electrical  tests  over  the  military  temperature 
range,  packaging  material  requirements,  lead  material  requirements, 
quality  conformance  inspection  requirements,  etc  The  difference 
tetween  a  MIL-STD-883  part  (or  SMD  part)  and  a  JAN  part  is  that  a 
government  approved  specification  is  not  listed  in  MIL-M  38510; 
therefore,  manufacfunng  line  certification  has  not  been  performed  for 
the  MIL-STD-883  part.  The  manufacturer  is  then  free  to  vary  the 
manufacturing  process  at  any  time.  This  could  then  change  the 
mechanical  or  electrical  characteristics  of  the  part 

Refers  to  parts  which  have  been  tested  to  a  vendor's  equivalent  MIL- 
STD-883  test.  The  vendor  may  or  may  not  modify  the  scope  of  the 
tests  and  a  careful  analysis  is  required  to  determine  |ust  how  similar 
to  MIL-STD-883  the  tests  are 
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DESIGN— TOPIC  D4 


Program  and  design  reviews  are  key  vehicles  for  measuring  development 
progress  and  preventing  costly  redesigns.  Participation  by  government  individuals 
knowledgeable  in  R&M  is  critical  to  provoking  discussions  that  bring  out  the  issues 
important  to  R&M  success.  Of  course,  the  questions  to  be  posed  to  the 
development  contractor  depend  on  the  timing  of  the  review  as  indicated  below. 
Action  Items  should  be  assigned  at  the  reviews  based  on  open  R&M  issues  and 
the  reliability  engineer  must  follow-up  to  ensure  that  they're  resolved  satisfactorily. 

Table  D4-1 :  Major  Program  Reviews 


Review  Purpoae  R&M  Engineers  Role 


System  Requirements 
Review  (SRR) 

To  ensure  a  complete 
understanding  of  system 
specification  and  statement  of 
work  requirements.  This  is  usually 
done  by  means  of  a  detailed 
expansion  and  review  of  the 
contractor's  technical  proposal 

Discuss  the  performance  of  all 
required  R&M  tasks  '"nd 
requirements  with  contractor  R&M 
personnel  Topics  such  as  the 
contractor  s  overall  reliability 
program  plan,  data  items  and 
delivery  schedule  are  usually 
discussed 

Preliminary  Design  Review 
(PDR) 

To  evaluate  progress  and 
technical  adequacy  of  the 
selected  design  approach  prior  to 
the  detailed  design  process 

Discuss  and  evaluate  R&M 
tradeoffs  and  reasoning  leading  to 
the  selected  design  approach 
Review  preliminary  Rf-M 
modeling,  allocations  and 
predictions  to  ensure  adequacy  in 
meeting  R&M  requirenients 
Discuss  status  of  other  R&M  tasks 
such  as  parts  control,  derating, 
thermal  design  and  reliability 
critical  items 

Critical  Design  Review 
(CDR) 

To  ensure  that  the  detailed  design 
satisfies  the  requirements  of  the 
system  specification  before 
freezing  the  design  for  production 
or  field  testing 

Review  the  final  reliability  analysis 
and  modeling  to  ensure  R&M 
requirements  are  met  Discuss 
parts  control  program  status  and 
military  part  procurement  lead 
time  requirements  Review 
adequacy  of  the  final  thermal 
analysis  and  derating  Discuss 
RSM  testing 

Te'  •  Readiness  Review 

(ThR) 

To  ensure  that  all  CDR  problems 
have  been  satisfactorily  resolved 
and  to  determine  if  the  design  is 
mature  enough  to  start  formal 
testing 

Review  R&M  test  plans  and 
procedures  to  ensure  acceptable 
ground  rules  and  compliance  with 
requirements 

Production  Readiness 
Review  (PRR) 

To  review  test  results  and 
determine  whether  or  not  the 
design  is  satisfactory  tor 
production 

Discuss  R&M  testing  results  and 
ensure  any  design  deficiencies 
found  during  testing  have  been 
corrected.  Discuss  production 
quality  assurance  measures  and 
burn-in  procedures  to  ensure 
adequacy 
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Table  D4-2:  Design  Review  Checklist 


OuMtion 


n«vl«w  Wh«r*  Usually 
Most  Applicabla 

SRR  POR  CDR  TRR  PRR  Ramartcs 


R&M  Management 

What  are  the  avenues  of  technical  X  X 
interchange  between  the  R&M  group 
and  other  engineering  groups  (e.g.. 

Design.  Systems  Engineering,  ILS. 
Procurement,  arKi  Test  and 
Evaluation)? 

Does  the  reliability  group  have  X  X 

membership  and  a  voice  in  decisions 
of  the  Material  Review  Board.  Failure 
Review  Board,  and  Engirteering 
Change  Review  Board’ 

Is  the  contractor  and  subconlractor(s)  X  X 
a  member  of  the  Government 
Industry  Data  Exchange  Program 
(GIDEP)’  What  is  the  procedure  for 
comparing  parts  on  the  ALERT  list  to 
parts  used  in  the  system? 

Are  reliability  critical  items  given  X 

special  attention  in  the  form  of  special 
analysis,  testing  or  destructive 
laboratory  evaluation? 

Do  the  purchase  orders  require  X  X 

vendors  to  deliver  specified  levels  ol 
R&M&T  based  on  allocation  of  higher 
level  requirements’ 

Does  the  reliability  group  have  X  X 

access  to  component  and  failure 
analysis  experts  and  how  are  they 
integrated  into  the  program? 

Is  there  adequate  communication  X  X 
between  testability  design  engineers 
and  the  electrical  design  group  to 
ensure  that  testability  considerations 
are  worked  into  the  upfront  design’ 

Are  JAN  microcircuits  (MIL-M-38510)  X 

and  semiconductors  (MlL-S-19500) 
being  used  wherever  possible  and 
are  procurement  lead  times  for  these 
devices  adequate’ 


R&M  engineering  shoutd  pertidpete 
at  ell  ei>gineering  group  meetings 
where  R&M  is  affected.  Easy 
avenues  of  technical  interchange 
between  the  eiectrical  design  group 
and  other  groups  such  as  thermal 
ertgineerlng  must  exist 

X  X  Membership  or  an  option  to  voice  an 
opinion  is  essential  if  the  failure 
tracking  artd  corrective  action  loop  is 
to  be  completed 

X  Incoming  part  types  should  be 

checked  against  the  GIDEP  ALERT 
data  base  and  incoming  ALERTS 
should  be  checked  against  the 
system  parts  list.  (GIDEP  ALERTS 
are  notices  of  deficient  parts, 
materials  or  processes ) 

X  Critical  parts  are  usually  defined  by 

contract  or  by  MIL-STD-705  Methods 
ol  tracking  critical  parts  must  be 
identified  by  the  contractor  See  Topic 
D5  tor  a  critical  items  checklist 

Requirements  should  require 
verification  by  analysis  or  lest 


X  Failure  analysis  is  essential  to 

determine  the  cause  and  effect  of 
failed  components 


X  Part  quality  in  order  ol  preference 

MIL-M-38510.  Class  B;  MIL-STD-883, 
Class  B;  MIL-STD-883  vendor 
equivalent,  commercial  hermetically 
sealed  JAN  parts  usually  require 
longer  procurement  times  (3  to  6 
months)  which  sometimes  causes 
commercial  parts  to  be  forced  into  the 
design 
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MM  Management  OuMtioiM 

Review  Where  Usually 

Most  Appllcabie 

SRR  FOR  COR  TRR  PRR  Remarlis 

Where  nonstandard  parts  are  used,  X  X 
are  they  procured  via  a  specification 
control  drawing  (SCO)  arid  do  they 
have  at  least  two  suppliers?  Are 
methods  for  nonstandard  part 
approval  clearly  establish^  and  Is 
there  a  dear  understarKfing  of  what 
constitutes  a  standard  and 
nonstandard  part? 

Has  an  up-tonjate  preferred  parts  X  X 
selection  list  (PPSL)  been 
established  for  use  by  designers? 


X  Specification  control  drawings  should 

spedty  reliability,  environment  and 
testing  requirements. 


OESC  and  DISC  establish  baseline 
PPSLs  which  should  be  the  basis  of 
the  contractor  s  list.  See  Topic  R6 
Note  2  for  points  of  contact 


R&M  Design 

Do  the  R&M&T  models  accurately  X  X 

reflect  the  system  configuration,  its 
modes  of  operation,  duty  cycles,  and 
implementation  of  fault  tolerance’ 

Do  predictions  meet  numerical  R&M  X  X  X 
spiecification  requirements?  Are 
prediction  procedures  in  accordance 
with  requirements? 

Have  R&M  allocations  been  made  to  X  X 
the  LRU  level  or  below?  Do  reliability 
predictions  compare  favorably  to  the 
allocation? 


If  not.  better  cooling,  part  quality  and 
or  redundancy  should  be  considered 


Weighted  reliability  allocations  should 
be  made  to  lower  levels  based  on  the 
upper  test  MTBF  (Ho),  or  similar 
measure 


Does  the  testability  analysis  show  X  X 

that  numerical  testability 

requirements  will  be  met  for  the 

organizational,  intermediate  and 

depot  repair  levels? 

Have  tradeoff  studies  been  X  X 

performed  in  the  areas  of  R&M&T? 


Has  a  thermal  analysis  been  X  X 

performed  to  ensure  an  adequate 
cooling  technique  is  used  and  have 
the  temperature  results  been  factored 
into  the  reliability  analysis? 

Has  piece  part  placement  been  X  X 

analyzed  to  ensure  that  high 
dissipating  parts  are  placed  away 
from  heat  sensitive  parts? 


If  nol,  alternate  design  concepts  must 
consider  including  more  automated 
features 


Typical  tradeoffs  might  include 
redundancy  levels,  weight,  power, 
volume,  complexity,  acquisition  cost, 
life  cycle  cost 

Thermal  analysis  is  essential  to  a 
complete  program. 


For  example,  high  power  dissipation 
components  such  as  large  power 
resistors,  diodes  and  transformers 
should  be  investigated. 


Have  methods  been  established  to  XX 
ensure  that  operating  temperatures  of 
off-the-shelf  equipment  will  be  within 
specified  limits? 


Reference  environmental 
requirements  in  the  system 
specification 
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Raviaw  Whart  Uaually 
Most  Applicabta 

OuMtiona  SRR  POR  COR  TRR  PRR  Remark* 


For  each  maintenance  level,  has  a  XX 

decision  been  made  for  each  item  on 

how  built-in-test,  automatic  test 

equipment,  and  general  purpose 

electronic  test  equipment  will  support 

fault  detection  and  isolation? 

Are  features  being  incorporated  into  X  X 

the  testability  design  to  control  false 

alarms? 


Typical  features  might  include 
definition  of  test  tolerances-  transient 
monitoring  and  control,  mu'tiple  run 
decision  logic,  environmental  effects 
filterir^  arKf  identification,  etc 


R&M  Testing 


Is  there  a  failure  reporting  and  X 

corrective  action  system  (FRACAS) 

in  place,  and  does  it  account  for 

failures  occurring  during  all  phases  of 

testing’’ 


X  X  FRACAS  should  include  data  from 
incoming  inspection,  development 
testir^g,  equipment  integrat  on  testing 
and  R&M  testing  FRACAS  should  be 
"closed  loop "  emphasizing  corrective 
action. 


Is  there  a  failure  analysis  capability 
and  will  failures  be  subjected  to  a 
detailed  analysis? 

Are  subcontractors  subjected  to  the 
same  FRACAS  requirements,  and 
will  their  failure  analysis  reports  be 
included  with  the  prime  contractor's 
reports? 

Does  the  reliability  demonstration  test 
simulate  the  operating  profile  seen  in 
the  field  and  will  all  modes  of 
equipment  operation  be  tested  over 
the  required  environmental 
extremes? 

Does  the  maintainability  and 
testability  demonstration  test  simulate 
realistic  failures  and  is  the  candidate 
task  list  sufficient  to  reduce  bias’’ 

Are  relevant  and  nonrelevant  failure 
definitions  clear  and  agreed  upon’’ 

Are  equipment  performance  checks 
to  be  performed  during  testing  clearly 
defined  and  has  the  information  to  be 
recorded  in  the  test  log  been  clearly 
defined  and  agreed  upon? 


XXX  Contractor  should  identify  criteria 

used  to  determine  which  failures  will 
be  analyzed 


X  X  The  test  must  simulate  the 

operational  profile  and  modes  to  have 
valid  results. 


X  X  Candidate  lists  should  be  four  to  ten 

times  the  size  of  the  test  sample 


X  X  See  Topic  T9  tor  failure  definitions 


X  Items  such  as  temperature  variations 
start  stop  of  vibration,  event 
occurrence  times  and  a  detailed 
description  of  system  recovery  after 
failure  should  be  included  as  a 
minimum. 
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DESIGN— TOPIC  D4 


Table  D4-2  (continued) 


R«vi«w  Wh«r«  Usually 
Moat  Applicable 

RAM  Oeaign  Oueationt  SRR  POR  COR  TRR  PRR  Remarfca 


Do  preliminary  plans  ior  ESC  meet  X  X  Temperature  and  random  vibration 

the  required  needs?  are  the  most  efiective  screens.  At 

module  level,  perform  20  to  40 
temperature  cycles  per  module.  At 
higher  levels,  perform  4  to  12  cycles 
(See  RADC-TR-82-87,  Stress 
Screening  of  Electronic  Hardware 
and  DOD-HDBK-344,  Environmental 
Stress  Screening  of  Electronic 
Equipment  and  Topics  T1 -T3  for 
guidance.) 


Note:  For  an  exhaustive  design  checklist  see  MIL-HDBK-338.  Chapter  7 
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Major  Concerns 


•  Has  the  contractor  developed  formal 
policies  and  procedures  for 
identification  and  control? 

•  Are  the  procedures  implemented  at 
the  initial  design  stage  and  do  they 
continue  through  final  acceptance 
period? 

•  Are  periodic  reviews  planned  to 
update  the  list  and  controls? 

•  Has  an  FMEA  been  performed  on 
each  critical  item? 


•  Are  compensating  features  included 
in  the  design? 

•  Does  the  contractor’s  control  plan 
eliminate  or  minimize  the  reliability 
risk? 


•  As  a  minimum,  are  the  following 
criticality  factors  considered: 

•  Failures  jeopardizing  safety 

•  Restrictions  on  limited  useful  life 

•  Design  exceeding  derating  limits 

•  Single  sources  for  parts 

•  Historically  failure  prone  items 

•  Stringent  tolerances  for 
manufacturing  or  performance 

•  Single  failure  points  that  disrupt 
mission  performance 


Comments 


•  Policies  should  be  distributed  to 
design,  manufacturing,  inspection 
and  test  personnel. 

•  The  program  has  to  start  early  so 
that  safety  related  items  can  be 
minimized. 

•  Reviews  at  SRR,  PDR,  and  CDR 
must  be  considered. 

•  Failure  modes  need  to  be  identified 
so  that  control  procedures  can  be 
developed. 

•  Features  such  as  safety  margins, 
overstress  testing,  special 
checkouts  should  be  considered. 

•  Development  of  a  list  of  critical  items 
is  only  half  the  solution:  controls 
such  as  stress  tests,  design 

.  margins,  duty  cycles,  and  others 
must  be  considered 

•  A  list  of  critical  items,  personnel 
responsible  for  monitoring  and 
controlling,  and  review  procedures 
must  be  established.  Other 
application  unique  critical  items 
should  be  identified  by  the  procuring 
activity. 
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Section  A 


What  R&M  ANALYSES  should  be  required  and  how  should 

they  be  evaluated? 

R&M  Engineer’s  Role 


A1  Select  Appropriate  Analyses .  63 

A2  Reliability  Prediction  Methods .  66 

A3  Maintainability  Prediction  Methods . 67 

A4  Testability  Analyses .  70 

R6  Specify  Contract  Data  Requirements .  28 

Evaluate  Deliverables  and  Analyze  R&M 

AS  Reliability  Analysis  Checklist .  71 

A6  Use  of  Existing  Reliability  Data .  72 

A7  Maintainability/Testability  Analysis  Checklist .  73 

A8  FMECA  Analysis  Checklist .  74 

A9  Redundancy  Equations .  75 

A10  Parts  Count  Reliability  Prediction .  79 

A11  Reliability  Adjustment  Factors .  91 
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For  More  Information 

MIL-STD-756  "Reliability  Modeling  and  Prediction” 

MIL-STD-1629  “Procedures  for  Performing  a  Failure  Mode,  Effects  ind 
Criticality  Analysis” 

MIL-HDBK-217  Prodiction  of  Electro'^*''  Fquipment" 

MIL-HDBK-472  “Maintainability  Prediction" 

RADC-TR-82-179  “Sneak  Analysis  Application  Guidelines" 
RADC-TR-87-55  “Predictors  of  Organizational-Level  Testability  Attributes" 
RADC-TR-77-287  “A  Redundancy  Notebook" 
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ANALYSIS— TOPIC  A1 


2 

le 

I 

a 


i  1  t 


s. 

c 


.5^ 

*0 


•c 

<0 

Q) 


I  i 


S’  f 

g  i  o 

s  <i>  <n 

& 


^  i 


I  f  ^ 

i!  « 


§ 

< 


? 

& 

<0 

*<5 

3  .S: 
4A  <Q 

>^5 

<T3  0) 
C  > 
<0 

.-S' « 

2  i 
(0  £ 
C  <Q 

l.a 
,£  « 
a  Oi 

2  o 


I 

.2 

(/) 

a> 

•D 

S  S 

(0  c 

c  ^ 
?  E 

0) 

*0 
Q> 

£ 
c 

■c 

CD 

a> 


K 


O 

t: 

& 


1 

a 


o 

2 


c 

o 

I « 

>•  K 

<fl  2 

<«  iS 

0)  p 
2  n 

CD  U> 
«  0) 

.£  S 
(0  (0 

II 

2  c 
*  o 

V) 

0)  JO 
£  0^ 

II 

S  c 


|«£ 

ll 

2  e 

|i 

"  cc 

I? 

5  § 
c  to 
(0  ^  . 

3  2  W> 

^  -n  O 
»  c  » 

l.i  i> 

CO  7  ^ 

S  ®  -o 

3  »  m 
S  ■=  w 

■£  I'S 

<S  ¥  w 
Q  SO 


(B  ® 

2- 

i  ° 
§  2 

1  S 

2i 
£  »6 
=  o: 

w  E 

.  to 

JS  V> 

Q)  >• 

-n  W 
O  to 
E  CO 

E.i 

2  m 
u»  to 

«  £ 
w>  v> 
to  to 

D  2 


c 

O) 

(A 

to 

•D 

o 


£  £ 


Vi 


I  f 


2  S 
•-  £ 
V>  U 

to  to 
«  2 
?  a 

m  CO 


to 

3 

to 

to 

1* 

(A 

•I- 

c 

to 

S 

O  crt 

•§1 

§  ® 

a| 

c  ^ 
5  £ 

11 


c 

o 


c 

o 


I 

< 


I 


RADC  RELIABILITY  ENGINEER  S  TOOLKIT 


63 


•  To  assist  in  the  efficient  design  of  BIT  and  FIT.  equipment. 

•  To  establish  and  rank  critical  failures  •  More  applicable  to  equipment  performing  critical 

T  .  .  .  .  .  functions  (e  g.,  control  systems). 

•  To  identify  interface  problems  ’  ' 


ANALYSIS— TOPIC  A1 
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linabllit^rediction  Methods 


ANALYSIS-TOPIC  A2 
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ANALYSIS— TOPIC  A3 


< 


I 


c 

1 5 

a.  a 


o 

o 

2 

0) 

w 

3 

5  cft 
cl 

■B  I 

CO  & 

8l 

o 

1o 


c 

o 

(A 


E  ^ 


CP 

£ 

(A 

S 

gf 

c 

(A 

2 

£ 

&& 

CO 

a 

to 

CP 

E 

oo 

p 

o 

»-  o 

fsi 

rp 

0) 

O) 

2 

> 

(0 

o 

c 

o 


3 

o 


0) 

u 

c 

(0 

c 

0} 

2 

<0  6 
2  ^ 


^  (A 

If  O 

O  <0  — 

•O  C 

?  £ 
o  ^ 
y  0) 


3 


CO 

Q  0) 

-  ,i 
»  i 

_  E 

CO  Ci 
W  00 

fli  >> 
o  </) 

c  *0 
2  c 
c  <0 

CP 

r 

=  CP 

if 


S  c 

I 

CP  5 
^*0  0 
o  =  U 


c  o  5  o 

d)  ^  ■P 


las 

W  CP  •Q 
^  U  CP 

•O  c 
<0  o 
1a  ^  Z. 
c 


CP 


CP  tn  iS 
C  CP  CO 
<  *0  "D 


T3 

2 

'5 

z 

a. 

C 

o 


(0 

E 

o 

c  ^ 
r  X) 

•O  ■= 
^  <0 
•o:  > 
c 

8  .52 


CP 

o 

c 

(0 

c 

£ 

c 

<5 

E 

CP 

_> 

o 

2  ‘ 
5< 
tr 

oS. 

u.  ^ 


.£  CO 
S'  CP 
5-0 

£- 

CO 


%Si 

I  S 

CO  ^ 

o 


2  -c 

3  .52 

c  2 
o  */P 

U  CP 


2 

3 

a 

y 

to 

O 

c 

OP 

(0 


O) 

c 


(0 

OP 

(/) 

CO 

CP 


^  .52  S 

CO  CO  2 

?•  t:  2 

0)  (0  ^ 

CC  Q.  O 


(/) 

8 

£ 

OP 

E 

O) 

c 

c 

3 

o 

2 


i:  u 


3 

O 


c  _ 
o  CO  £ 

.=  iS  0) 

t>  ®  c 

5 

O  </>  3  (0 

o  S| 

®  5.  o  .c 

I  ®  ?:s 
il2| 

Q*  E  CP  *“ 
<||o 

<0  5  c  P 

Q-  E  CO 


CO 

c 

0) 

£ 

O) 

-8  g, 

5?  CO 
£  to 
ctP  i- 

^  Ss 
^  OP 

S'  ^ 

CL  (i> 
<  ”0 


<  i 


3  b 

P  CP 
CP  o 

Q.  C 


OP 

>1 
B  o 


(O 


o 

u 


z  c 

CO  - 
C  3 
CP  o 
C  *= 

CO  E  c 

CP  CO  CO 

2  E  E 


>  i 

Is 

>  C 
OP  CO 

- 

CO  £  c 

CP  CO  CO 

2  E  E 


3 

O 


CO 


CO 


CL  E 


^  2 

5  t3 

CO  c> 

E  *5 

CO  0) 

S  o 

">  ■‘s 

*  c 
£  TO 


W  w 

II 


C 

CO 

u 

£:  CP 

V)  CP 

■o  o 

CO  p 
Cp 

gs 

3  © 
crj2 
CP  CO 


“U  2 
.  CP  CP 
CO  > 
y  3  CP 

c  2  = 

CO  0^  CO 

■C  ^  c 

g^B 

p  s  u  . 

o  "g  J  . 

=  1": 
£  e  " 

TO  Q.  « ■ 
C  in  ^ 

TO  E  i 

£  S  i  ■ 

(C  in 

E  «  2 


RADC  RELIABILITY  ENGINEER  S  TOOLKIT 


67 


Procedur*  Purpose  Application  Basic  Parameters  of  Measure  Information  Required 
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ib... 


■-•■jSlli 


Major  Concerns 


CommanU 


Models 


Are  all  functional  elements  included  in  the  reliability 
block  diagrams/model? 

Are  all  modes  of  operation  considered  in  the  math 
model? 


System  design  drawings/diagrams  must  be 
reviewed  to  be  sure  that  the  reliability  model ' 
diagram  agrees  with  the  hardware. 

Duty  cycles,  alternate  paths,  degraded 
conditions  and  redundant  units  must  be  defined 
and  modeled 


Do  the  math  model  results  show  that  the  design  Unit  failure  rates  and  redundancy  equations  are 
achieves  the  reliability  requirement?  used  from  the  detailed  part  predictions  in  the 

system  math  model. 


Allocation 

Are  system  reliability  requirements  allocated 
(subdivided)  to  useful  levels? 


Does  the  allocation  process  consider  complexity, 
design  flexibility,  and  safety  margins? 

Prediction 

Does  the  sum  of  the  parts  equa'  the  value  of  the 
module  or  unit’ 


Are  the  environmental  conditions  and  part  quality 
representative  of  the  requirements? 


Are  the  circuit  and  part  temperatures  defined  and 
do  they  represent  the  design’ 


Are  equipment,  assembly,  subassembly  and  part 
reliability  drivers  identified? 


Are  part  failure  rates  from  acceptable  sources  (i.e  . 
MIL-HDBK-217)’ 

Is  the  level  of  detail  for  the  part  failure  rate  models 
sufficient  to  reconstruct  the  result? 


Are  critical  components  such  as  VHSIC.  Monolithic 
Microwave  Integrated  Circuits  (MMIC).  Application 
Specific  Integrated  Circuits  (ASIC)  or  Hybrids 
highlighted? 


Useful  levels  are  defined  as:  equipment  for 
subcontractors,  assemblies  for  sub¬ 
subcontractors.  circuit  boards  for  designers 

Conservative  values  are  needed  to  prevent 
reallocation  at  every  design  change 


Many  predictions  conveniently  neglect  to 
include  all  the  parts  producing  optimistic  results 
(check  for  solders  connections,  connectors, 
circuit  boards) 

Optimistic  quality  levels  and  favorable 
environmental  conditions  are  often  assumed 
causing  optimistic  results. 

Temperature  is  the  biggest  driver  of  part  failure 
rates:  low  temperature  assumptions  will  cause 
optimistic  results 

Identification  is  needed  so  that  corrective 
actions  for  reliability  improvement  can  be 
considered 

Use  of  generic  failure  rales  require  submission 
of  backup  data  to  provide  credence  m  the 
values 

Each  component  type  should  be  sampled  and 
failure  rales  completely  reconstructed  for 
accuracy 

Prediction  methods  for  advanced  parts  should 
be  carefully  evaluated  for  impact  on  the  module 
and  system 
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System  development  programs  often  make  use  of  existing  equipment  (or 
assembly)  designs,  or  designs  adapted  to  a  particular  application.  Sometimes, 
lack  of  detailed  design  information  prevents  direct  prediction  of  the  reliability  of 
these  items  making  use  of  available  field  and/or  test  failure  data  the  only  practical 
way  to  estimate  their  reliability.  If  this  situation  exists,  the  following  table 
summarizes  the  information  that  is  desired. 


Table  A6-1 :  Use  of  Existing  Reliabiiity  Data 


Information  Required 

Equipment 
Field  Data 

Equipment 
Teat  Data 

Piece  Part 
Data 

Data  collection  time  period 

x 

X 

X 

Number  of  operating  hours  per  equipment 

Total  number  of  part  hours 

X 

X 

X 

Total  number  of  observed  maintenance  actions 

X 

Number  of  "no  defect  found"  maintenance  actions 

X 

Number  of  induced  maintenance  actions 

X 

Number  of  "hard  failure"  maintenance  actions 

X 

Number  of  observed  failures 

X 

X 

Number  of  relevant  failures 

X 

X 

Number  of  nonrelevant  failures 

X 

X 

Failure  definition 

X 

X 

Number  of  equipment  or  parts  to  which  data  pertains 

X 

X 

X 

Similarity  of  equipment  of  interest  to  equipment  for  which  data 
is  available 

X 

X 

Environmental  stress  associated  with  data 

X 

X 

X 

Type  of  testing 

X 

Field  data  source 

X 
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:0/. 


M^or  Concerns 


Are  the  maintainability /testability  prediction 
techniques  and  data  used  dearly  described? 

Is  there  a  dear  description  of  the  mairttenance 
concept  and  all  ground  rule  assumptions? 


Are  worksheets  provided  which  show  how  LRU 
repair  times  were  arrived  at? 

Are  step-by-step  repair  descriptions  provided  to 
back  up  repair  time  estimates? 

Are  fault  isolation  time  estimates  realistic? 


Are  fault  isolation  ambiguity  levels  considered  in 
the  analysis? 

Can  repair  times  be  reconstructed  from  the 
worksheets  and  is  addition,  subtraction, 
multiplication  and  division  correct? 

Are  preventive  maintenance  tasks  described? 

Is  all  the  equipment  included  in  the  prediction? 

Has  the  best  procedure  been  seleded  to  provide 
estimates  for  the  testability  attributes? 


Comments 


Repair  level,  LRU/module  definition,  spares 
availability  assumptions,  test  equipment 
availability  assumptions,  tools  availability 
assumptions,  personnel  assumption, 
environmental  corKlitions. 

The  breakout  of  repair  time  should  include;  fault 
isolation,  disassembly,  interchange,  reassembly 
arid  checkout. 


Overestimating  BIT/FIT  capability  is  the  primary 
cause  of  optimistic  repair  time  estimates 


Checking  is  mundane  but  often  results  in  errors 
and  irKonsistencies  being  found 

This  includes  frequency,  maintenance  time  and 
detailed  task  description 


Because  of  the  number  of  variables  which  affect 
testability  and  the  number  of  different 
procedures  available  to  effect  analyses, 
there  must  be  rationale  ar)d  logic  provided  to  explain 
why  ttie  particular  approach  was  taken. 


Are  the  numerical  values  of  the  testability  attributes 
within  specified  tolerances? 

Does  the  test  equipment,  both  hardware  and  All  test  points  should  be  accessible 

software,  meei  all  design  requirements? 

Are  the  simulation  and  emulation  procedures  to  be 
used  to  simulate/emulate  units  of  the  system,  for 
diagnostics  development,  reasonable  and 
practical? 
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Maior  Conctmt 


CommanU 


Is  a  system  definition/description  provided  and  is  it 
compatible  with  the  system  specification? 

Are  ground  rules  clearly  stated? 


Are  block  diagrams  provided  showing  functional 
dependencies  at  all  equipment  indenture  levels? 


Does  the  failure  effect  analysis  start  at  the  lowest 
hardware  level  and  systematically  work  to  higher 
indenture  levels’’ 

Are  failure  mode  data  sources  fully  described? 

Are  detailed  FMECA  worksheets  provided?  Do  the 
worksheets  clearly  track  from  lower  to  higher 
hardware  levels?  Do  the  worksheets  clearly 
correspond  to  the  block  diagrams  provided? 

Are  failure  severity  classes  provided? 


Are  results  clearly  summarized’ 

Are  results  ol  FMECA  being  used  to  impact  other 
program  design  decisions? 


These  include  approach,  failure  definition, 
acceptable  degradation  limits,  level  of  analysis, 
etc. 

This  diagram  should  graphically  show  what 
items  (parts,  circuit  cards,  subsystems,  etc.)  are 
required  for  the  successful  operation  of  the  next 
higher  assembly 

The  analysis  should  start  at  the  lowest  level 
specified  in  the  SOW  (e  g.,  part,  circuit  card, 
subsystem,  etc ) 

Consider  use  of  MIL-HDBK-338  failure  mode 
data 

Worksheets  should  provide  an  item  name, 
indenture  code  item  function,  list  of  item  failure 
modes,  effect  on  next  higher  assembly  and 
system  for  each  failure  mode,  and  a  criticality 
ranking 

Typical  classes  are; 

1  Catastrophic  (life  death) 

2  Critical  (mission  loss) 

3  Marginal  (mission  degradation) 

4  Minor  (maintenance/repair) 

Actions  lor  risk  reduction  of  single  point  failures, 
critical  Items,  areas  needing  BIT/FIT,  etc. 

BIT  design,  critical  parts,  reliability  prediction, 
derating,  fault  tolerance 
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Many  military  electronic  systems  readiness  and  availability  requirements  exceed 
the  level  of  reliability  to  which  a  serial  chain  system  can  be  practically  designed. 
Use  of  high  quality  parts,  a  sound  thermal  design  and  extensive  stress  derating 
may  not  be  enough.  Fault  tolerance,  or  the  ability  of  a  system  design  to  tolerate  a 
failure  or  degradation  without  system  failure,  is  required.  The  most  common  form 
of  fault  tolerance  is  redundancy  where  additional,  usually  identical,  units  are  added 
to  a  system  in  parallel  with  the  other  units.  Because  this  situation  is  very  common, 
the  reliability  equations  for  common  redundancy  situations  are  included  below. 

The  following  represents  a  sample  list  of  specific  redundancy  relationships  which 
define  failure  rate  as  a  function  of  the  specific  type  of  redundancy  employed.  For  a 
more  comprehensive  treatment  of  redundancy  concepts  and  the  reliability 
improvements  achievable  through  their  applications  see  R  ADC-TR-77-287,  “A 
Redundancy  Notebook.” 
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Table  A9>1 :  Redundancy  Equation  Approximations  Summary 


Redundwwy  Equations 

With  Repair 

Without  Repair 

All  units  are  active  on-line  with  equal  unit 
failure  rates,  (n  -q)/n  required  for 
success. 

Equation  1  n!(X)‘'*’ 

Equation  4 

(ri-q-l)IW 

i  X 

i«n-Q  1 

Two  active  on-line  units  with  different 
failure  and  repair  rates.  One  of  two 
required  for  success. 

Equation  2 

^  _  ^a^b[(m-a  +  M-e)  +  (^A  +  ^b)] 

Equation  5  x  -i-  x  x  2 

^  ^  ''A  ''B  ^  '^A^B 

(M-a){m-b)  +  (Pa  +  Pb)(>^a  +  ^b) 

Xa*  Xb^  +  X^Xb 

One  standby  off-line  unit  with  n  active  on¬ 
line  units  required  for  success.  Off-line 
spare  assumed  to  have  a  failure  rate  of 
zero.  On-line  units  have  equal  failure 
rates. 

Equations  n|nx  +  (l -P)u|x  Equation  $ 

ii  +  n(P+1)X  P  +  1 


Key: 

K, ,  is  the  effecttve  failure  rate  of  the  redundant  configuration  where  x  ol  y  units  are  required  tor  success 
n  =  number  of  active  on-line  units,  n!  is  n  factorial  (eg  .  5!  =  5x4x3x2xl  =  120, 1 1  =  1, 0!  =  1 ) 

K  =  failure  rate  of  an  individual  on-line  unit  (failures'^hour) 

q  =  number  ol  on-line  active  units  which  are  allowed  to  fail  without  system  failure 

^  =  repair  rate  (ti  =  1  /Mo,  where  Mo  is  the  mean  corrective  maintenance  time  in  hours) 

P  =  probability  switching  mechanism  will  operate  properly  when  needed  (P  =  t  with  perfect  switching) 

Notes: 

1  Assumes  all  units  are  functional  at  the  start 

2  The  approximations  represent  time  to  first  failure 

3  CAUTION:  Redundancy  equations  tor  repairable  systems  should  not  be  applied  if  delayed  maintenance  is  used 
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Example  1 : 

A  system  has  five  active  units,  each  with  a  failure  rate  of  220  f/10^  hours,  and  only 
three  are  required  for  successful  operation.  If  one  unit  fails,  it  takes  an  average  of 
three  hours  to  repair  it  to  an  active  state.  What  is  the  effective  failure  rate  of  this 
configuration? 

Solution: 

Substituting  the  following  values  into  Equation  1 : 
n  =  5 
q  =  2 
=  1/3 

^(5-2)5  ~  ^3  5 

^  5!  (220  X  10 
~  {5-2-1)!(1/3)* 

X35  =  5.75x10'®f/hour=.00575f/10®hours 


Example  2: 

A  ground  radar  system  has  a  2  level  weather  channel  with  a  failure  rate  ot  50  f/10® 
hours  and  a  6  level  weather  channel  with  a  failure  rate  of  180  f/10®  hours.  Although 
the  6  level  channel  provides  more  comprehensive  coverage,  the  operation  of 
either  channel  will  result  in  acceptable  system  operation.  What  is  the  effective 
failure  rate  of  the  two  channels  if  one  of  two  are  required  and  the  M^,  is  1  hour? 

Solution: 

Substituting  the  following  values  into  Equation  2; 

=  50x10® 

Xb  =  180x  10-® 

Ha  =  =  1/Mct  =  1 

_  (50x10  ®)(180x  10  -®)  [(1  -H)  +  (50x  10-6+ 180x10  ®)] 

“  (1)(1)  +  (1 +  1)(50x10  ®  +  180x10-®) 

X,2  =  1.8x  10-®f/hour=.018f/10®hours 
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Example  3: 

Determine  the  effective  failure  rate  for  8  of  10  identical  units  required  with  no  repair. 
The  failure  rate  of  a  single  unit  is  60  f/10^  hours. 

Solution: 

Substituting  the  following  values  into  Equation  4: 
n  =  10 
q  =  2 

\  =  60x10-« 

,  _  60x10  « 


60x10® 

^8  10  = 

1  1  J_ 

8  9  ^  10 

>^810  =  1.79  X  10  '*f/hour=  179  f/10®  hours 

Having  two  redundant  units  improves  the  system  failure  rate  from  480  f/10®  hours 
(8  units  required  x  60  f/10®  hours  each)  to  179  f/10®  hours. 
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A  Standard  technique  for  predicting  reliability  when  detailed  design  data  such  as 
part  stress  levels  is  not  yet  available  is  the  parts  count  reliability  prediction 
technique.  The  technique  has  a  “built-in"  assumption  of  average  stress  levels 
which  allows  prediction  in  the  conceptual  stage  or  source  selection  stage  by 
estimation  of  the  part  types  and  quantities.  This  section  contains  a  summary  of  the 
MIL-HDBK-217  technique  for  four  of  the  most  common  operational  environments: 
space  flight  (Sf).  ground  fixed  (Gf),  Airborne  Inhabited  Cargo  (A,c)  and  Airborne 
Uninhabited  Fighter  (Auf)-  All  failure  rates  in  the  following  tables  are  in  terms  of 
failures  per  million  hours. 

Assuming  a  series  reliability  model,  the  equipment  failure  rate  can  be 
expressed  as: 


n 

^EOUIP  ~  ^  NAgi'^Toi 
1=  1 

Where 

^Eouip  =  total  equipment  failure  rate  (failures/10®  hrs) 

Xq,  =  generic  failure  rate  for  the  ith  generic  part  type  (failures/ 10®  hrs) 
■no.  =  quality  factor  for  the  ith  generic  part  type 
N,  =  quantity  of  the  ith  generic  part  type 
n  =  number  of  different  generic  part  types 
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Table  A10-1  (continued) 


ANALYSIS— TOPIC  A10 
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Table  A10-2:  Semiconduc  or  Failure  Rates  (x 


ANALYSIS— TOPIC  A10 
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ANALYSIS-  TOPIC  A10 
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Plastic  CFR  55514  .0041  .00>9  .015 


Table  A10-3  (continued) 


ANALYSIS— TOPIC  A10 
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Conventional  Pulsed  230  470  930  7000 


Table  A10-4  (continued) 


ANALYSIS— TOPIC  A10 
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ANALYSIS— TOPIC  A10 
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ANALYSIS— TOPIC  All 
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ANALYSIS— TOPIC  All 


ANALYSIS— TOPIC  A1 1 


Temperature  Conversion  Factors 

For  each  10°C  decrease  in  part  ambient  temperature  multipiy  the  system 
series  MTBF  by  1.25  and  for  each  10°C  increase  muitiply  by  0.8. 


Adjustment  Examples 

Example  1  (Quaiity  Adjustment): 

An  equipment  has  been  designed  using  “typical  military"  part  quality  levels  and 
has  an  MTBF  of  400  hours.  What  would  the  expected  reliability  be  if  all  “vendor 
equivalent"  quality  parts  were  substituted? 

Solution:  400  hours  x  .2  =  80  hours. 

Example  2  (Environmental  Adjustment): 

An  equipment  designed  for  use  in  a  Ground  Mobile  environment  has  an  MTBF  of 
100  hours.  What  would  be  the  equipment’s  expected  MTBF  if  operated  in  a  Ground 
Benign  environment? 

Solution:  100  hours  x  7  =  700  hours. 

Example  3  (Temperature  Adjustment): 

An  equipment  has  an  MTBF  of  60  hours  with  its  current  cooling  supply.  A  potential 
reallotment  of  cooling  air  would  decrease  the  equipment  average  part  ambient 
temperature  by  12°C.  How  would  the  equipment  MTBF  change? 

12°C 

in°r 

Solution:  60  hours  X  (1.25)  =78.4  hours 


If  this  same  equipment  were  to  experience  a  1 2®  C  increase  in  temperature  due  to 
less  cooling  air,  the  estimated  MTBF  would  be: 

-12°C 

60  hours  X  (1.25)  =  45.9  hours 
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Section  T 


What  are  the  types  of  R&M  TESTING  and  how  do  I  structure  an 

effective  test  program? 

R&M  Engineer’s  Role: 


Develop  a  Tailored  Test  Program 

T1  ESS  Process .  97 

T2  ESS  Placement .  98 

T3  “R&M  2000”  ESS .  99 

T4  RGT  and  RQT  Application .  101 

Demonstration  Plan  Selection 

T5  Reliability .  102 

T6  Maintainability .  104 

T7  Testability .  105 

Review  Plans/Procedures 

T8  Review  FRACAS .  106 

T9  Reliability  Demonstration  Plan  Checklist .  108 

T10  Reliability  Test  Procedure  Checklist .  112 

T11  Maintainability  Test  Plan  Checklist .  113 

T1 2  R&M  Tes^  7articipation  Criteria .  115 

T13  Review  R&M  Demonstration  Reports .  116 
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For  More  Information 

MIL-STD-471  "Maintainability  Verification/Demonstration/Evaluation" 

MIL-STD-781  "Reliability  Testing  for  Engineering  Development, 
Qualification  and  Production" 

MIL-HDBK-781  "Reliability  Test  Methods,  Plans,  and  Environments  for 

Engineering  Development,  Qualification,  and  Production" 


DOD-HDBK-344  "Environmental  Stress  Screening  of  Electronic  Equipment" 

MIL-HDBK-189  "Reliability  Growth  Management" 

RADC-TR-84-25  "Reliability/Maintainability  Operational  Parameter 
Translation"  (Volumes  I  and  II) 

RADC-TR-86-241  "Built-ln-Tesf  Verification  Technique'- 
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TESTING— TOPIC  T1 


Environmental  Stress  Screening  (ESS)  has  been  the  subject  of  many  recent 
studies.  RADC’s  position  has  been  that  no  one  set  of  generic  screens  is  best  for 
every  situation.  Determination  of  the  optimum  screens  for  a  particular 'oduct,  built 
by  a  particular  manufacturer,  at  a  given  time  is  an  iterative  process.  Procedures  for 
planning  for  and  controlling  the  screening  process  are  contained  in  DOD-HDBK- 
344  (USAF)  Environmental  Stress  Screening  of  Electronic  Equipment.  The 
process  cpn  be  depicted  as  shown  below: 


Figure  T1  -1 ;  ESS  Process 


RADC  RELIABILITY  ENGINEER  S  TOOLKIT 


97 


TESTING— TOPIC  T2 


Topic  T2^SS  PlScemiRT 

Level  of 

Assembly 

Advantages 

Disadvantages 

Assembly 

•  Cost  per  flaw  precipitated  is 

•  Test  detection  efficiency  is 

lowest  (unpowered  screens). 

relatively  low. 

•  Small  size  permits  batch 

•  Test  equipment  cost  tor 

screening. 

•  Low  thermal  mass  allows 
high  rates  of  temperature 
change. 

•  Temperature  range  greater 
than  operating  range 
allowable. 

screens  is  high. 

Unit 

•  Relatively  easy  to  power  and 

•  Thermal  mass  precludes  high 

monitor  performance  during 

rates  of  change  or  requires 

screen. 

costly  facilities. 

•  Higher  test  detection 

•  Cost  per  flaw  significantly 

efficiency  than  assembly 

higher  than  assembly  level. 

level. 

•  Temperature  range  reduced 

•  Assembly  interconnections 
(e.g.,  wiring  backplane)  are 
screened. 

from  assembly  level 

System 

•  All  potential  sources  of  flaws 

•  Difficult  and  costly  to  test  at 

are  screened. 

temperature  extremes. 

•  Unit  interoperability  flaws 

•  Mass  precludes  use  of 

detected. 

effective  vibration  screens  or 

•  High  test  detection  efficiency. 

makes  use  costly. 

•  Cost  per  flaw  is  highest. 
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TESTING— TOPIC  T3 


Screen  Type,  Parameter 

Assemblies  (Printed 

Equipment  or  Unit 

and  Conditions 

Wiring  Assemblies)  (SRU)* 

(LRU/LRM) 

Thermal  Cycling  Screen 

Temperature  Range 

From  -  54°C  to  -i-  85°C 

From  -54°Cto  ^7rC 

(Minimum)  (See  Note  1 ) 

Temperature  Rate  of 

30°C/Minute  (Chamber 

5°C  (Chamber  Air  Temp) 

Change  (Minimum) 

Air  Temp) 

(See  Note  2) 

Temperature  Dwell 

Until  Stabilization 

Until  Stabilization 

Duration  (See  Note  3) 

Temperature  Cycles 

25 

10 

(Minimum) 

Power  On/Equipment 

No 

(See  Note  5) 

Operating 

Equipment  Monitoring 

No 

(See  Note  6) 

Electrical  Testing  After 

Yes  (At  Ambient 

Yes  (At  Ambient 

Screen 

Temperature) 

Temperature) 

Random  Vibration 
(See  Note  7) 

Acceleration  Level 

(See  Note  8) 

6  G  rms 

Frequency  Limits 

50-1000  Hz 

Axes  Stimulated  Serially  or 

2  (minimum) 

Concurrently 

(See  Note  9) 

Duration  of  Vibration 
(Minimum) 

•  Axes  stimulated  serially 

10  Minutes^Axis 

•  Axes  stimulated 

10  Minutes 

concurrently 

Power  On/Equipment 

(See  Note  5) 

Equipment  Monitoring 

(See  Note  6) 

Piece  Parts:  Begin  the  manufacturing  and  repair  process  with  100  defects  per  million  or 

less  (see  note  10). 

*  SRU — Shop  Replaceable  Unit  LRM — Line  Replaceable  Module 

LRU — Line  Replaceable  Unit 
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TESTING— TOPIC  T3 


Notes: 

1 .  Temperatures  beyond  stated  values  are  acceptable. 

2.  Rapid  transfers  of  the  equipment  between  one  chamber  at  maximum  temperature  and 
anotfter  chamber  at  minimum  temperature  are  acceptable.  SRU  temperature  rates  of 
change  may  be  reduced  if  equipment  damage  will  occur  at  30°C/minute. 

3.  The  temperature  has  stabilized  when  the  temperature  of  the  part  of  the  test  item 
considered  to  have  the  longest  thermal  lag  is  changing  no  more  than  2°C  per  hour. 

4.  A  minimum  of  5  thermal  cycles  must  be  completed  after  the  random  vibration  screen. 
Random  vibration  frequently  induces  incipient  failures. 

5.  Shall  occur  during  the  low  to  high  temperature  excursion  ot  the  chamber  and  during 
vibration.  When  operating,  equipment  shall  be  at  maximum  power  loading.  Power  will 
be  OFF  on  the  high  to  low  temperature  excursion  until  stabilized  at  the  low  temperature. 
Power  will  be  turned  ON  and  OFF  a  minimum  of  three  times  at  temperature  extremes 
on  each  cycle. 

6.  Instantaneous  go/no-go  performance  monitoring  during  the  stress  screen  is  essential 
to  identify  intermittent  failures  when  power  is  on. 

7.  Specific  level  may  be  tailored  to  individual  hardware  specimen  based  on  vibration 
response  survey  and  operational  requirements 

8.  When  random  vibration  is  Sholied  at  the  equipment  level,  random  vibration  is  not 
required  at  the  subassembly  .ovel.  However,  subassemblies  purchased  as  spares  are 
required  to  undergo  the  same  random  vibration  required  for  the  equipment  level.  An 
"LRU  mock-up"  or  equivalent  approach  is  acceptable. 

£  One  axis  will  be  perpendicular  to  plane  of  (he  circuit  board(s)  'LRM(s). 

1 0.  The  Air  Force  or  its  designated  contractor  MAY  AUDIT  part  defective  rates  at  its 
discretion.  The  test  procedure  will  include  thermal  cycling  as  outlined  below.  Sample 
sizes  and  test  requirements  are  included  in  t^*?  Strec:  Screening  Military  Handbook, 
DOD-HDBK-344(AF). 

Minimum  Temperature  Range  From  -  54°C  to  -i-  100°C 

Minimum  Temperature  Rate  of  Change  The  total  transfer  time  from  hot-to-cold  or 

cold-to-hot  shall  not  exceed  one  minute. 
The  working  zone  recovery  time  shall  be 
five  minutes  maximum  after  introduction  of 
the  load  from  either  extreme  in  accordance 
with  MIL-STD-B83C. 

Until  Stabilization  (See  Note  3) 

25 

No 

Yes  (At  high  and  low  temperatures) 


Temperature  Dwell 
Minimum  Temperature  Cycles 
Power  On 'Equipment  Monitoring 
Electrical  Testing  After  Screen 


100 


RADC  RELIABILITY  ENGINEER  S  TOOLKIT 


TESTING— TOPIC  T4 


The  Reliability  Qualification  Test  (RQT)  is  an  “accounting  task"  used  to  measure 
the  reliability  of  a  fixed  design  configuration.  It  has  the  benefit  of  holding  the 
contractor  accountable  some  day  down  the  road  from  his  initial  design  process.  As 
such,  he  is  encouraged  to  seriously  carry  out  the  other  design  related  reliability 
tasks.  The  Reliability  Growth  Test  (RGT)  is  an  “engineering  task"  designed  to 
improve  the  design  reliability.  It  recognizes  that  the  drawing  board  design  of  a 
complex  system  cannot  be  perfect  from  a  reliability  point  of  view  and  allocates  the 
necessary  time  to  fine  tune  the  design  by  finding  problems  and  designing  them  out. 
Monitoring,  tracking  and  assessing  the  resulting  data  gives  insight  into  the 
efficiency  of  the  process  and  provides  nonreliability  persons  with  a  tool  for 
evaluating  the  development’s  reliability  status  and  for  reallocating  resources  when 
necessary.  The  forms  of  testing  serve  very  different  purposes  and  complement 
each  other  in  development  of  systems  and  equipments.  An  RGT  is  not  a  substitute 
for  an  RQT,  or  other  reliability  design  tasks 


Table  T4-1 :  RGT  and  RQT  Applicability  as  a  Function  of 
System/Program  Constraints 


Reliability  Growth  Teat 

Reliability  Qualification  Test 

System/Program  Parameter 

Apply 

Consider  Don't  Apply 

Apply 

Consider  Don't  Apply 

Challenge  to  s;ale-of-the-art 

X 

X 

Severe  use  environment 

X 

X 

One-of-a-kind  system 

X 

X 

High  quantities  to  be  produced 

X 

X 

Benign  use  environment 

X 

X 

Critical  mission 

X 

X 

Design  flexibility  exists 

X 

X 

No  design  flexibility 

X 

X 

Time  limitations 

X 

X 

Funding  limitations 

X 

X 

Very  high  MTBF  system 

X 

X 
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iflability  Demoristratibh  f’lah  Selection 


TESTING— TOPIC  T5 
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TESTING— TOPIC  T5 
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TESTING— TOPIC  T6 


Program  Conatraintr 

Teet 

Characteristic 

Calantiar  Time 
Required 

Number  of 

Equlpmente 

Available 

Test  Facility 
Limitations 

Laval  of 

Maintainability 

Raquirad 

Desirad 
Confidence  in 
Results 

Fixed  sample 

Much  less  than 

No  effect  on 

No  effect  on 

Fixed  sample 

size  or 

that  required 

sample  size 

sample  size 

size  test  gives 

sequential  type 
tests. 

for  reliability 
demo.  Time 
required  is 
proportional  to 
sample  size 
number. 

Sample  size 
may  vary 
depending  on 
program. 

number. 

number. 

demonstrated 
maintainability 
to  desired 
confidence. 
Sequential  is 
test  of 
hypothesis. 

Test  plan  risks 

Low  producer 

Must  have 

No  effect  on 

Higher 

(consumer  and 

and  consumer 

ability  to 

sample  size 

confidence 

producer)  risks  require 

(1  -  consumer  larger  sample 

risk  =  confidence)  sizes  than 

Risks  can  tje  higher  risks. 

tailored  to 

program 

simulate 

operational 

maintenance 

environment, 

scenario,  skills 

levels 

available. 

number. 

levels  require 
more  samples 
than  lower 
confidence 
levels 

Note;  Demonstration  facility  must  have  capacity  tor  insertion  of  simulated  faults 
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TESTING— TOPIC  T7 


Program  Conatrainu 


-toat 

Ctiaractariatic 

Calendar  Tima 
Required 

Number  of 

Equipments 

Available 

liaat  Facility 
Limitattons 

Desired 
Confidence  In 
Results 

Fixed  sample 

Calendar  time 

No  effect  on 

Same  as  that 

Provides  for 

size  type  tests 

much  less  than 

sample  size 

required  for 

producers 

that  required 

number. 

maintainability 

risks  of  10%. 

for  reliability 

demonstration. 

Provides 

demonstration. 

consumer 

Time  required 

assurance  that 

is  proportional 

designs  with 

to  sample  size 

significant 

May  vary 

deviations  from 

depending  on 

specified 

program. 

values  will  be 

rejected. 

Preset  Risks 

Risks  inversely 

(consumer  anr| 

p^oportioiial  to 

producer) 

sample  size 

(1  -  consumer 

used. 

risk  =  confidence) 


Notes: 

1  Sample  si2e  aependent  on  loial  number  of  sample  maintenance  tasks  selected  as  pe'  paragraph  A  10  <t  o( 
MIL-STD-471A 

2  Demonstration  facility  must  have  capability  tor  insertion  of  simulated  laults 
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TESTING— TOPIC  T8 


Early  elimination  of  failure  trends  is  a  major  contributor  to  reliability  growth  and 
attaining  the  needed  operational  reliability.  To  be  effective,  a  clused  loop 
coordinated  process  must  be  implemented  by  the  system/equipment  contractor.  A 
description  of  the  major  events  and  the  participant  s  actions  is  shown  below. 


Ev«nt  Functions  Actions 


Failure  or  Malfunction 

]  Operators 

•  Identify  a  problem,  call  for  maintenance,  annotate 

the  incident 

1 

1 

t 

Maintenance 

•  Corrects  the  problem,  logs  the  failure 

1 _ 

Quality: 

•  Inspects  the  correction 

Failure  Report 

Maintenance; 

•  Generates  the  lailure  report  with  supporting  data 

(time,  place,  equipment,  item,  etc  ) 

1 

r 

Quality 

•  Insures  completeness  and  assigns  a  travel  tag  lor 
the  failed  item  for  audit  control 

Data  Logged 

R&M 

*  Log  all  the  failure  reports,  validate  the  tailures  and 

r 

forms,  classify  the  failures  (inherent,  induced, 
false  alarm) 

Failure  Review 

R&M 

•  Determine  failure  trends  (l  e  .  several  laiiures  of 

1 

1 

the  same  or  similar  part) 

L _ 

Design 

•  Review  operating  procedures  for  error 

Failure  Analysis 

R&M 

•  Decide  which  parts  will  be  destructively  analyzed 

1 

r 

Physics  of  Failure 

•  Perform  failure  analysis  to  determine  the  cause  ol 
failure  (i.e  .  pan  or  external) 

1 

Quality 

•  Inspect  incoming  test  data  lor  the  part 

Failure  Correction 

Design 

•  Redesign  hardware,  if  necessary 

*  Mow  p^^ri  c*'’  'est  procedure 

r 

Quality 

•  Evaluate  incoming  test  procedures,  inspect 
redesigned  hardware. 

Post  Data  Review 

R&M 

•  Close  the  loop  by  collecting  and  evaluating  post 

lest  data  lor  reoccurrence  of  the  failure 


Figure  T8-1 :  Failure  Reporting  System  How  Diagram 
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TESTING  — TOPIC  T8 


Table  T8-1 :  FRACAS  Evaluation  Checklist 


Topic 

Item*  to  B«  AddrasMd 

General 

•  Closed  loop  (i.e  .  reported,  analyzed,  corrected  and  verified) 

•  Responsibility  assigrted  for  each  step 

•  Overall  control  by  one  group  or  function 

•  Audit  trail  capability 

•  Travel  tags  for  c"  tailed  iics-.c. 

•  Fast  turn-around  for  analysis 

Failure  Report 

•  Clear  descnption  of  each  event 

•  Surrounding  cortditions  noted 

•  Operating  time  indicated 

•  Maintenance  repair  times  calculated 

•  Built-in-test  indications  stated 

Failure  Analysis 

•  Perform  if  three  or  more  identical  or  similar  parts  laii 

•  Perform  if  unit  reliability  is  less  than  hall  ol  predicted 

•  Results  should  indicate  overstress  condition,  manut  icturmg 
delect,  adverse  environmental  condition,  mamtenan.  >  induced  or 
wearout  failure  mode 

Failure  Daia 

•  Ccllaled  by  w^ek  and  month  by  unit 

•  Compared  to  allocated  values 

•  Reicbility  growth  tracked 

•  Problems  indicated  and  tracked 

•  Correction  data  collected  for  verification 
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Topic 


Itama  to  B«  Addroasad 


Purpose  and  Scope 


Relerence  Documents 
Test  Facilities 


Test  Requirements 


Test  Schedule 


Test  Conditirr.s 


•  Statement  o<  overall  test  ob)ectives 

•  General  descnption  of  all  tests  to  be  performed 

•  List  all  applicable  relerence  documents 

•  Descnption  of  test  item  configuration 

•  Sketches  of  system  layout  dunng  testing 

•  Senal  numbers  of  units  to  be  tested 

•  General  description  of  lest  facility 

•  Test  safety  features 

•  Identification  of  test  locatior^ 

•  General  description  of  failure  analysis  facility 

•  Security  of  test  area 

•  Security  of  test  equipment  and  records 

•  Test  safety  provisions 

•  Pre-reliability  environmental  stress  screening  (ESS) 

•  Test  length 

•  Number  of  units  to  be  tested 

•  Number  of  allowable  failures 

•  Descnption  of  MIL-HDBK-7Bf  test  plan  showing  accept,  reject  and 
continue  test  requirements 

•  List  of  government  furnished  equipment 

•  List  and  schedule  of  test  reports  to  be  issued 

•  Start  date  (approximate) 

•  Finish  date  (approximate) 

•  Test  program  review  schedule 

•  Number  of  test  hours  per  day 

•  Number  of  test  days  per  week 

•  Description  of  thermal  cycle 

•  Description  of  thermal  survey 

•  Description  of  vibration  rurvey 

'  Description  of  unit  under  test  mounting  method 

•  Description  of  test  chamber  capabilities 

•  List  of  oil  limited  life  items  and  iheir  expected  life 
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Topic  Itotna  to  Be  Addreaaed 

•  Descnption  of  all  preventive  maintenance  tasks  and 
their  frequency 

•  Description  of  unit  under  lest  calibration  requirements 

•  Description  of  unit  under  test  duly  cycle 

•  General  descnption  ol  unit  under  test  operating  mode  and 
exercising  method. 

Test  Monilonng  •  Description  of  test  software  and  software  verification  lelhod 

•  List  of  all  units  under  test  functions  to  be  monitored  arid  monitoring 
method 

•  List  of  all  test  equipment  parameters  to  be  monitored  and 
monitoring  method 

•  Method  and  frequency  of  recording  all  monitored  parameters 

Test  Participation  •  Description  of  all  contractor  functions 

•  Description  of  all  contractor  responsibilities 

•  Description  of  all  government  responsibilities 

•  Description  of  lest  management  structure 

Failure  Definitions  The  following  typies  of  failures  should  be  defined  as  rek-.'ant  m  the 

test  plan: 

•  Design  defects 

•  Manufactunng  defects 

•  Physical  or  functional  degradation  below  specilicalior  limits 

•  Intermittent  or  transient  failures 

•  Failures  of  limited  life  parts  which  occur  before  the  specified  life  ot 
the  part 

•  Failures  which  cannot  be  attributed  to  a  specific  cause 

•  Failure  of  built-in-test  (BIT) 

The  following  types  of  failures  snould  be  defined  as  non'elevant  m 
the  test  plan 

•  Failures  re.sulting  from  improper  installation  or  handing 

•  Failure  of  instrumentation  or  monitoring  equipment  w  iich  is 
external  to  equipment  under  test 

•  Failures  resulting  from  overstress  beyond  specificati;  n  limits  due 
to  a  test  facility  fault 

•  Failures  resulting  from  procedural  error  by  technicians 

•  failures  induced  by  repair  actions 

•  A  secondary  failure  which  is  the  direct  result  of  a  faiiye  of  another 
part  within  the  system 
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Topic 

Itoma  to  Be  Addreaaad 

Test  Ground  Rules  Thie  following  test  ground  Oiles  should  be  stated  in  the  test  plan 

•  Transient  Faihires — Each  transient  or  intermittent  failure  is  to  be 
counted  as  relevant.  If  several  intermittent  or  transient  failures  can 
be  directly  attributed  to  a  sirfgle  hardware  or  software  malfunction 
which  is  corrected  arul  verified  during  the  test,  then  only  a  sirrgle 
failure  will  be  counted  as  relevant. 

•  Classification  of  Failures — All  failures  occurring  during  reliability 
testing,  after  contractor  failure  analysis,  shall  be  classified  as  either 
relevant  or  nonrelevant.  Based  on  the  failure  analysis,  the 
contractor  shall  justify  the  failure  as  relevant  or  nonrelevant  to  the 
satisfaction  of  the  procuring  activity. 

•  Pattern  Failure — A  pattern  failure  is  defined  as  three  or  more 
relevant  failures  of  the  same  part  in  identical  or  equivalent 
applications  whose  95th  percentile  lower  confidence  limit  failure 
rate  exceeds  that  predicted. 

•  Malfunctions  Observed  During  Test  Set  Up,  TroubleshooUng  or 
Repair  Verification — Malfunctions  occurring  during  test  set  up. 
troubleshooting  or  repair  verification  tests  shall  not  be  considered 
as  reliability  test  failures:  however,  such  malfunctions  shall  be 
recorded  and  analyzed  by  the  contractor  to  determine  the  cause  of 
malfunctions  and  to  identify  possible  design  or  part  deficiencies 

•  Test  Time  Accumulation — Only  the  time  accumulated  during  the 
equipment  power  on’  portion  of  the  test  cycle  shall  be  considered 
as  test  time,  provided  that  all  functions  are  operating  as  required 
Operating  time  accumulated  outside  the  operational  cycles  such 
as  during  tests  performed  to  check  out  the  setup  or  to  verify  repairs 
shall  not  be  counted  Also,  time  accumulated  during  degraded 
modes  of  operation  shall  not  be  counted 

•  Design  Changes  to  the  Equipment: 

1 .  After  test  re/ect  decision — With  procuring  activity  approval,  the 
equipment  may  be  redesigned  and  retested  from  time  zero 

2.  Major  design  change  prior  to  test  reject — The  contractor  may 
stop  the  test  for  purposes  of  correcting  a  major  problem  The 
test  will  restart  from  time  zero  after  the  design  change  has  been 
made. 

3.  Minor  design  change  prior  to  test  reject — With  procuring 
activity  approval,  the  test  may  be  halted  for  the  purpose  of 
making  a  minor  design  change  Test  time  will  resum  ^  from  the 
point  at  which  it  was  stopped  and  the  design  change  shall  have 
no  effect  on  the  classification  of  previous  failures.  Minor 
changes  made  as  a  result  of  other  testing  may  be  incorporated 
with  procuring  activity  approval,  without  declaring  a  failure  of 
the  equipment  under  test. 

•  Failure  Categorization — In  order  to  clearly  evaluate  test  results 
and  identity  problem  areas,  failure  causes  will  be  categorized  as 
( 1 )  deficient  system  design,  (2)  deficient  system  quality  control, 
and  (3)  deficient  part  design  or  quality. 
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Topic 

Items  to  Be  Addressed 

Test  Logs  Tl .  a  following  types  of  test  logs  should  be  described  in  the  test  plan : 

•  Equipment  Data  Sheets  which  will  be  used  to  record  the  exact 
values  of  all  parameters  measured  during  functional  testing  of  the 
equipment. 

•  Jast  Log  which  is  a  comprehensive  narrative  record  of  the  required 
test  events.  All  names  and  serial  numbers  of  the  equipments  to  be 
tested  shall  be  listed  before  start  of  the  test.  An  entry  shall  be  made 
in  the  test  log  each  time  a  check  is  made  on  the  equipment  urxfer 
test,  including  data,  time,  elapsed  time,  and  result  (e  g.,  pass/ 
malfunction  indication/failure  or  etc.).  An  entry  shall  be  made  in  the 
log  whenever  a  check  is  made  of  the  test  facilities  or  equipments 
(such  as  accelerometers,  thermocouples,  input  power,  self-test, 
etc  ).  In  the  event  of  a  failure  or  malfunction  indication,  all  pertinent 
data,  such  as  lest  conditions,  facility  conditions,  test  parameters 
and  failure  indicators,  will  be  recorded.  The  actions  taken  to  isolate 
and  correct  the  failure  shall  also  be  recorded.  Whenever 
engineering  changes,  or  equipment  changes  are  implemented,  an 
entry  shall  be  made  in  the  log 

•  Failure  Summary  Record— the  failure  summary  record  must 
chronologically  list  all  failures  that  occur  during  the  test.  This  record 
must  contain  all  the  information  needed  to  reach  an  accept  or 
reject  decision  for  the  test  Each  failure  must  be  described  and  all 
failure  analysis  data  must  be  provided. 

•  Failure  Report— lor  each  failure  that  occurs,  a  failure  report  must 
be  initiated.  The  report  should  contain  the  unit  that  failed,  serial 
number,  time.  data,  symptoms  of  failure  and  part  or  parts  that 
failed. 


'Most  ol  these  contents  also  apply  to  reliability  growth  testing 
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^pic  TIP:  Reliability 


lure  Checklist 


Topic 


Item*  to  Be  Addressed 


Equipment  Operation 


On/Off  Cycle 
Operation  Modes 


Exercising  Methods 


Performance  Verification 
Procedure 


Failure  Event  Procedure 


Adjustments  and  Preventive 
Maintenance 


A  general  description  of  the  equipment  under  test  and  its  operation 
must  be  provided. 

Specific  on/off  times  for  each  subsystem  must  be  describi  d 

Specific  times  of  operation  for  each  system/subsystem  mode  must 
be  described 

Methods  of  exerasing  all  system  subsystem  operation  modes  must 
be  described.  (Note  The  system  should  be  exercised  continuously, 
not  just  power  on.) 

Step  tjy  step  test  procedures  must  be  provided  which  fully  describe 
how  and  when  each  performance  parameter  will  be  measured. 
Acceptable  and  unacceptable  limits  of  each  measured  paiameter 
should  also  be  specified  All  failure  and  out-of-tolerance  indicators 
must  be  described  and  their  location  defined  Programmable  alarm 
thresholds  must  be  specified 

Step  by  step  procedures  must  describe  specific  actions  to  be  taken 
in  the  event  of  a  trouble  indication 

Step  by  step  procedures  must  be  provided  which  fully  describe  how 
and  when  all  adjustments  and  preventive  maintenance  actions  will 
be  performed. 
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Topic  Itomt  to  B«  AddrotMd 

Purpose  and  Scope  •  Statement  of  general  test  objectives 

•  General  description  of  test  to  be  performed 

Reference  Documents  *  List  of  all  applicable  reference  documents 

Test  Facilities  *  Description  of  test  item  configuration 

•  Sketches  of  system  layout  during  testing. 

•  Serial  numbers  of  units  to  be  tested 

•  General  description  of  test  facility. 

•  Test  safety  features 

•  Identification  of  test  location 

•  Description  of  all  software  to  be  used,  both  internal  and  external  to 
the  equipment  under  test. 

•  Security  of  test  area. 

•  Security  of  test  equipment  and  records 

•  Description  of  all  lest  equipment  to  be  used 

Test  Requirements  •  Description  of  MlL-STD-471  test  plan  requirements. 

•  Method  of  generating  candidate  fault  list. 

•  Method  of  selecting  and  injecting  faults  from  candidate  list 

•  List  of  government  furnished  equipment. 

•  List  and  schedule  of  test  reports  to  be  issued 

•  Levels  of  maintenance  to  be  demonstrated 

•  Spares  and  other  support  material  requirements 

Test  Schedule  •  Start  date  (approximate). 

•  Finish  date  (approximate). 

•  Test  program  review  schedule 

Test  Conditions  •  Description  of  environmental  conditions  under  which  test  will  be 

performed. 

•  Modes  of  equipment  operation  during  testing 

Test  Monitoring  •  Method  of  monitoring  and  recording  test  results 

Test  Participation  •  Test  team  members  and  assignments. 

•  Test  decision  making  authorly 
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Topic 

Item*  to  B«  Addressed 

Test  Ground  Rules  with 

•  Instrumentation  failures. 

Respect  to 

•  Maintenance  due  to  secorvLiry  failures 

•  Technical  manual  usage  and  adequacy. 

•  Maintenance  inspection. 

•  GFE  usage. 

•  Maintenance  time  limits. 

•  Skill  level  of  maintenance  technicians. 

Testability  Demonstration 
Considerations 

•  Repair  levels  for  which  requirements  will  be  demonstrated. 

•  Built-in-test  requirements  to  be  demonstrated 

•  External  tester  requirements  to  be  demonstrated 

•  Evaluation  method  for  making  pass/fail  decision 

•  Performance  of  FMEA  prior  to  test  start 

•  Method  of  selecting  and  simulating  candidate  faults 

•  Acceptable  levels  of  ambiguity  at  each  repair  level 
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Degree  of  Participation 


Depends  on: 


•  Availability  of  program  resources  to  support  on-siie  personnel. 

•  How  important  R&M  are  to  program  success. 

•  Availability  and  capability  of  other  government  on-site  personnel  (i.e.,  Defense 
Ck>ntract  Administrative  Service  (DCAS),  Air  Force  Plant  Representative  Office 
(AFPRO),  Naval  Plant  Representative  Office  (NAVPRO),  etc.). 

•  Confidence  in  and  credibility  of  contractor. 


Test  Preliminaries 

•  All  test  plans  and  procedures  must  be  approved. 

•  Agreements  must  be  made  among  government  personnel  with  respect  to 
covering  the  test  and  incident  reporting  procedures. 

•  Units  under  test  including  serial  numbers  should  be  documented. 

•  Test  equipment  including  serial  numbers  should  be  documented. 

•  Working  fire  alarms,  heat  sensors  and  overvoltage  alarms  should  be  used. 

•  Trial  survey  runs  should  be  made  per  the  approved  test  plan. 

Test  Conduct 

•  Approved  test  plans  and  procedures  must  be  available  and  strictly  adhered  to. 

•  Equipment  must  not  be  tampered  with. 

•  Test  logs  must  be  accurately  and  comprehensively  maintained  as  in 
plan/procedures. 

•  Appropriate  government  personnel  must  be  kept  informed 
(per  established  agreements). 

•  Only  authorized  personnel  should  be  allowed  in  area  (a  list  should  be  posted). 

•  Test  logs,  data  sheets,  and  failure  reports  should  be  readily  available  for 
government  review. 

•  Units  under  test  should  be  sealed  to  prevent  tar  ipering  or  unauthorized  repair. 

•  A  schedule  of  inspections  and  visits  should  be  maintained. 

•  No  repairs  or  replacements  should  be  made  without  a  government  witness. 

•  Government  representatives  must  take  part  in  failure  review  process. 

•  Failed  items  should  have  "travel  tags"  on  them. 

•  Technical  orders  should  be  used  for  repair  if  available. 
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ibpicTtlS^eli^Iit^ai 
>ports^K|g!^ 


iShtalriabllUyl^'^p'nstradofV 

4, 


•  Identification  and  description  of  equlpment/system  tested 

•  Demonstration  objectives  and  requirements 

•  Test  Plans 

•  Test  Risks 

•  Test  Times 

•  Test  Conditions 

•  Test  Facilities 

•  Data  Analysis  Techniques 

•  Statistical  Equations 

•  Accept/Reject  Criteria 

•  Test  Results  (Summarized) 

Reliability 

•  Test  Hours 

•  Number  of  Failures/Number  of  Incidents 

•  Classification  of  Failures 

•  Data  Analysis  Calculations 

•  Application  of  Accept/Reject  Criteria 

•  Failure  Trends/Design  and  Process  Deficiencies 

•  Status  of  Problem  Corrections 
Maintainability 

•  Maintenance  Tasks  Planned 

•  Maintenance  Tasks  Selected 

•  Selection  Method 

•  Personnel  Qualifications  Performing  Tasks 

•  Documentation  Used  During  Maintenarice 

•  Measured  Repair  Times 

•  Data  Analysis  Calculation 

•  Application  of  Accept/Reject  Criteria 

•  Discussion  of  Deficiencies  Identified 
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Testability 

•  Summary  data  for  each  item  involved  in  testability  demonstration  including 
original  plans,  summarized  results  and  any  corrective  action  taken. 

•  Recommended  action  to  be  taken  to  remedy  testability  deficiencies  or  improve 
the  level  of  testability  achievable  through  prime  equipment  engineering 
changes,  ATE  improvements  and/or  test  program  set  improvements. 

Deviations  from  Test  Plan/Procedures  and  Risk  Assessment 

Data 

•  Test  Logs 

•  Failure  Reports 

•  Failure  Analysis  Results 
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OPERATIONAL  PARAMETER  TRANSLATION 


BfeCuuse  field  operation  introduces  factors  which  are  uncontrollable  by  contractors 
(e.g.,  maintenance  policy),  “contract”  reliability  is  not  the  same  as  “operational" 
reliability.  For  that  reason,  it  is  often  necessary  to  convert,  or  translate,  from 
“contract"  to  “operational"  terms  and  vice  versa.  This  appendix  is  based  on 
RADC-TR-84-25,  "Reliability/Maintainability  Operational  Parameter  Translation." 
Table  t.1  defines  the  R&M  parameters  applicable  to  the  translation  models 
summarized  in  Tables  1.2  through  1.4  for  three  of  the  more  common  environments 
encountered:  ground  fixed  (Gp),  airborne  inhabited  transport  (A,t)  and  airborne 
uninhabited  fighter  (Aof)  (as  defined  in  MIL-HDBK-217).  RADC  has  a  study 
underway  to  develop  more  refined  models  considering  a  wider  range  of 
operational  and  design  influencing  variables  in  addition  to  addressing  a  wider 
range  of  environments.  Results  are  expected  around  September  1988. 


Table  1.1 :  Operational  Parameter  Translation  Definitions 


MTBM-TOT/'L 


Total  Operating  Hours 
Total  Maintenance  Events 


MTBM-INHERENT 


Total  Operating  Hours _ 

Inherent  Maintenance  Events 


MTBM-NO  DEFECT 


Total  Operating  Hours _ 

Total  Base  Level  No  Defect  Maintenance  Events 


MMH/PH-INHERENT 


Base  Level  Inherent  Maintenance  Manhours 
Possessed  Hours 


MMH/PH-TOTAL 


Total  Base  Level  Maintenance  Manhours 
Possessed  Hours 


M-MMH  TO  REPAIR 


Total  Base  Level  Corrective  Maintenance  Manhours 
Total  on  Equipment  Maintenance  Events 


MMH/FH  TOTAL 


Total  Base  Level  Maintenance  Manhours 
Flight  Hours 


Notes: 

1  MTBM  Mean-Time-Between-Maintenance 

MMH  PH:  Marntenance  Manhours  per  Possessed  Hour 
M-MMH;  Mean  Maintenance  Manhours 
MMH/FH:  Maintenance  Manhours  per  Flight  Hour 

2  Inherent  maintenance  events  are  those  events  caused  by  design  or  manutactunng  delects 

3  MTBM  Total  consists  ot  inherent,  induced  and  no  detect  found  actions 
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Examples  of  Operational  Parameter  Translation: 

Example  1.  Contract  to  operational  reliability,  a  program  manager  is 
requested  to  provide  an  estimate  of  the  total  maintenance  frequency  load  on  a 
certain  equipment  to  be  installed  in  a  fighter  aircraft  to  assist  in  projecting  base 
level  maintenance  requirements.  The  equipment  bay  is  not  environmentally 
controlled  resulting  in  a  classification  of  the  environment  as  airborne  uninhabited 
fighter  (Ayp).  A  reliability  prediction  of  the  series  MTBF  has  been  developed  with  a 
value  of  1000  hours.  What  MTBM-TOTAL  can  be  expected  in  the  field? 

Solution:  From  Table  1.4  the  equation  needed  is:  MTBM-TOTAL  =  .59  (MTBF) 
Since  a  1000  hour  MTBF  is  within  the  upper  and  lower  model  limits  of  65  hours  and 
1800  hours,  respectively,  the  translation  can  be  performed  and  results  in  an 
MTBM-TOTAL  of  74  hours. 

Example  2.  Operational  to  contract  reliability— Ground  Fixed  (possessed 
hours  are  greater  than  operating  hours).  A  ground  fixed  equipment  is  to  be 
replaced.  The  new  equipment  must  meet  or  exceed  a  field  MTBM-INH  of  1000 
hours,  which  is  based  on  a  system  life  of  possessed  hours,  io  ensure  improvement 
of  the  current  system's  field  performance.  The  current  system,  as  well  as  the  new 
system,  will  be  on-line  approximately  50  percent  of  its  possessed  time  (i.e  , 
operating  hours  less  than  possessed  hours).  The  program  manager  is  given  the 
task  of  identifying  a  series  MTBF  requirement  that  will  ensure  meeting  the  MTBM- 
INH  goal. 

Solution:  Since  the  system  is  only  on-line  performing  its  intended  function  50 
percent  of  the  time,  the  adjustment  steps  listed  in  Row  1  of  Table  1.2A  are  used 
since  the  translation  is  from  MTBM  to  MTBF.  Since  the  system  is  operated  one 
hour  for  every  two  possessed  hours,  the  ratio  of  possessed  hours  to  operating 
hours  is  21.  The  mean-possessed-time-between-maintenance  (1000  hours)  is 
then  divided  by  this  ratio  to  yield  an  MTBM-INH  of  500  hours.  From  Table  1.2  the 
appropriate  equation  is  MTBF  =  .028  (MTBM-INH)’”.  Since  an  MTBM-INH  of  500 
hours  is  within  the  model  limits  of  275  hours  and  2600  hours,  the  translation  can  be 
performed  to  yield  a  specification  MTBF  requirement  of  355  hours. 
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Table  1.2:  Summary  of  Ground  Fixed  Operational  Parameter  Translations 


OPERATIONAL  PARAMETER  TRANSLATION 
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Table  1.2A:  Possessed  to  Operating  Time  Adjustment  Steps  for  Ground  Fixed  Environment 


OPERATIONAL  PARAMETER  TRANSLATION 
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MTBF  to  Maintenance  Manhours  per  Operating  Hour  No  ad|ustment  necessary. 


Table  1.3:  Summary  of  Airtxyrne  Inhabited  Transport  Operational  Parameter  Translations 


OPERATIONAL  PARAMETER  TRANSLATION 
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Table  1.4:  Summary  of  Airtx)me  Uninhabited  Fighter  Operational  Parameter  Translations 


OPERATIONAL  PARAMETER  TRANSLATION 
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example  R4M  REQUIREMENT  PARAGRAPHS 


Example  Reliability  Requirements  for  the  System  Specification 


R.1  Reliability  Requirements 

Guidance:  The  use  of  the  latest  versions  and  notices  of  all  military  specifications, 
standards  and  handbooks  should  be  specified. 

Guidance:  When  specifying  an  MTBF,  it  should  be  the  “upper  test  MTBF  (no)”  as 
defined  in  MIL-STD-781.  When  specifying  MTBCF,  the  maintenance  concept 
needs  to  be  clearly  defined  for  purposes  of  calculating  reliability  of  redundant 
configurations  with  periodic  maintenance.  If  immediate  maintenance  wilt  be 
performed  upon  failure  of  a  redundant  element  then  specifying  the  system  MTTR 
is  sufficient,  if  maintenance  is  deferred  when  a  redundant  element  fails,  then  the 
length  of  this  deferral  period  should  be  specified. 

R.1.1  Mission  Reliability.  The  (system  name)  shall  achieve  a  mean-time- 

between-critical-failure  (MTBCF)  of _ hours  under  the  worst  case 

environmental  conditions  specified  herein.  MTBCF  is  defined  as  the  total  uptime 
divided  by  the  number  of  critical  failures  that  degrade  full  mission  capability  (FMC). 
FMC  is  that  level  of  performance  which  allows  the  system  to  perform  its  primary 
mission  without  degradation  below  minimum  levels  stated  herein.  For  purposes  of 
analyzing  redundant  configurations,  calculation  of  MTBCF  shall  reflect  the 
expected  field  maintenance  concept. 

R.1.2  Basic  Reliability.  The  (system  name)  shall  achieve  a  series  configuration 

mean-time-between-failure  (MTBF)  of _ hours  under  the  worst  case 

environmental  conditions  specified  herein.  The  series  configuration  MTBF  is 
defined  as  the  total  system  uptime  divided  by  the  total  number  of  part  failures. 

R.1.3  Reliability  Configuration.  The  reliability  requirements  apply  for  the 
delivered  configuration  of  the  system.  Should  differences  exist  between  this 
configuration  and  a  potential  production  configuration,  all  analyses  shall  address 
the  reliability  effects  of  the  differences. 

Guidance:  If  Equipment  or  system  performance  criteria  are  not  stated  elsewhere 
in  statement  of  work  or  specification,  the  following  paragraph  must  be  included. 

R.1.4  Reliability  Performance  Criteria.  The  minimum  performance  criteria  that 
shall  be  met  for  full  mission  capability  of  the  (system  name)  system  is  defined  as 
(specify  full  mission  capability). 

R.1.5  Reliability  t>esign  Requirements.  Design  criteria  and  guidelines  shall  be 
developed  by  the  contractor  for  use  by  system  designers  as  a  means  of  achieving 
the  required  levels  of  reliability. 

Guidance:  For  more  critical  applications,  level  2  or  1  derating  should  be  specified. 
See  Topic  D1  for  derating  level  determination.  Baseline  thermal  requirements  such 
as  ambient  and  extreme  temperatures,  pressui  e  extremes,  mission  profile  and 
duration,  temperature/pressure  rates  of  change  and  maximum  allowable 
temperature  rise  should  be  specified. 
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R.1.5.1  Thermal  Management  and  Derating.  Thermal  management  (design, 
analysis  and  verification)  shall  be  performed  by  the  contractor  such  that  the 
reliability  quantitative  requirements  are  assured.  RADC-TR-82-172,  "RADC 
Thermal  Guide  for  Reliability  Engineers,”  shall  be  used  as  a  guide.  Derating 
criteria  shall  be  established  for  each  design  such  that  all  parts  used  in  the  system 
are  derated  to  achieve  reliability  requirements.  As  a  minimum,  Level  3  of  AFSC 
Pamphlet  800-27  “Part  Derating  Guidelines"  shall  be  used  for  this  design. 

Guidance:  If  the  system  is  for  airborne  use,  MIL-STD-5400  must  be  referenced  in 
place  of  MIL-E-4158  (ground  equipment). 

R.1.5.2  Parts  Selection.  All  parts  employed  in  the  manufacture  of  the  system 
shall  be  selected  from  the  government  generated  and  maintained  Program  Parts 
Selection  List  (PPSL),  Electrical/Electronic  Parts  and  the  PPSL  for  Mechanical 
Parts.  Parts  not  covered  by  the  above  referenced  PPSLs  shall  be  selected  in 
accordance  with  MIL-E-4158  and  MIL-STD-454  and  require  approval  by  the 
procuring  activity. 

a.  Microcircuits.  Military  standard  microcircuits  must  be  selected  in  accordance 
with  Requirement  64  of  MIL-STD-454.  All  non-JAN  devices  shall  be  tested  in 
accordance  with  the  Class  B  screening  requirements  of  MIL-STD-883, 
Method  5004  and  5008,  as  applicable.  All  device  types  shall  be  tested  to  the 
quality  conformance  requirements  of  MIL-STD-883,  Method  5005  and  5008 
Class  B. 

b  Semicor)ductors.  Military  standard  semiconductors  must  be  selected  in 
accordance  with  Requirement  30  of  MIL-STD-454.  All  non-JANTX  devices 
shall  be  screened  in  accordance  with  Table  (I  of  MIL-S-19500.  All  device 
types  shall  be  tested  to  the  Group  A.  Table  III  and  Group  B,  Table  IV  quality 
conformance  requirements  of  MIL-S-19500,  as  a  minimum.  The  following 
device  restrictions  apply: 

(1 )  Only  solid  glass  metallurgically  bonded  axial  lead  diodes  and  rectifiers 
shall  be  used. 

(2)  TO-5  packages  shall  be  limited  to  the  solid  metal  header  type. 

(3)  All  semiconductor  device  junctions  must  be  protected  and  no  organic  or 
desiccant  materials  shall  be  included  in  the  package. 

(4)  Devices  using  aluminum  wire  shall  not  use  thermocompression  wedge 
bonding. 

(5)  Aluminum  TO-3  packages  shall  not  be  used. 

(6)  Germanium  devices  shall  not  be  used. 

c.  Electrostatic  Sensitive  Parts.  Certain  types  of  integrated  circuits  are 
susceptible  to  electrostatic  discharge  damage.  Appropriate  discharge 
procedures  are  necessary  when  handling,  storing  or  testing  these  parts  and 
design  selections  of  desired  devices  should  include  a  consideration  of  the 
effectiveness  of  the  input  or  other  protective  elements  included  in  the 
device  design. 
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R.1.6  Reliability  Test  and  Evaluation.  The  quantitative  reliability  levels  required 
by  paragraph  (R.1 )  shall  be  verified  by  the  following; 

R.1.6.1  The  final  approved  reliability  analyses  for  the  various  configurations  and 
worst  case  environments  shall  demonstrate  compliance  with  the  quantitative 
requirements  cited  in  paragraph  (R.1 ). 

R.t6.2  The  contractor  shall  demonstrate  that  the  reliability  (mission  and.'or  basic) 
requirements  have  been  achieved  by  conducting  a  controlled  reliability  test  in 
accordance  with  MIL-HDBK-781  Test  Plan  (specify  MIL-HDBK-781  Test  Plan.  See 
Topic  T5  for  Plan  Selection.)  The  lower  test  (MTBCF  and/or  MTBF)  to  be 

demonstrated  shall  be _ hours  tested  in  a _ environment.  Relevant 

failures  are  defined  as  any  malfunction  which  causes  loss  or  degradation  below 
the  performance  level  specified  for  the  (equipment/system)  and  can  be  attributed 
to  design  defect,  manufacturing  defect,  workmanship  defect,  adjustment, 
deterioration  or  unknown  causes.  Nonrelevant  failures  are  failures  caused  by 
installation  damage,  external  test  equipment  failures,  mishandling,  procedural 
errors,  dependent  failures  and  external  prime  power  failures. 

Guidance:  A  growth  test  may  apply  if  the  next  phase  is  production.  If  one  is 
required,  it’s  appropriate  to  require  a  higher  risk  (e.g.,  30  percent)  demonstration 
test.  See  RADC-TR-84-20  “Reliability  Growth  Testing  Effectiveness,”  Topic  T4 
and  Appendix  6  for  further  guidance. 

R.  1.6.3  The  contractor  shall  conduct  a  controlled  fixed  length  dedicated  reliability 

growth  test  of _ hours  using  MIL-HDBK-189  as  a  guide.  The  test  shall  be  at  the 

same  environmental  conditions  as  the  RQT  Although  there  is  no  pass  fail  criteria, 
the  contractor  shall  track  the  reliability  growth  process  to  ensure  improvement  is 
taking  place  by  effective  implementation  of  corrective  action. 

Guidance:  See  Electronic  Systems  Division  TR-85-148,  "Derated  Application  of 
Parts  for  ESD  Systems  Development”  (Attachment  2)  for  a  recommended  derating 
verification  procedure. 

R.1.6.4  The  contractor  shall  verify  the  thermal  and  electrical  stresses  on - 

percent  (3  to  5  percent  sample  is  typical)  of  the  semiconductor  and  microcircuit 
parts  by  measurement  while  the  equipment  is  operated  at  the  worst  case 
environment,  duty  cycle  and  load.  The  results  of  the  measurements  shall  be 
compared  to  the  derating  requirements  and  the  verification  shall  be  considered 
successful  if  measured  values  are  less  than  specified  derated  levels. 
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Example  Reliability  Requirements  for  the  Statement  of  Work 


R.2  Reliability  Program  Tasks 

R.2.1  Reliability  Program.  The  contractor  shall  conduct  a  reliability  pro()ram  in 
accordance  with  MIL-STD-785  including  the  following  tasks  as  a  minimum  to 
assure  reliability  consistent  with  state-of-the-art. 

R.2.2  Subcontractor  Control.  The  contractor  shall  establish  management 
procedures  and  design  controls  including  allocation  of  requirements  in  accordance 
with  Task  102  of  MIL-STD-785  which  will  insure  that  products  obtained  from 
subcontractors  will  meet  reliability  requirements. 

R.2.3  Reliability  Design  Reviews.  The  status  of  the  reliability  design  shall  be 
addressed  at  all  internal  and  external  design  reviews.  Task  103  of  MIL-STD-785 
shall  be  used  as  a  guide. 

R.2.4  Failure  Reporting,  Analysis  and  Corrective  Action  System 
(FRACAS).  The  contractor  shall  establish,  conduct  and  document  a  closed  loop 
failure  reporting,  analysis  and  corrective  action  system  for  all  failures  occurring 
during  system  debugging,  checkout,  engineering  tests  and  contractor 
maintenance.  Failure  reports  shall  be  retained  by  the  contractor  and  failure 
summaries  provided  to  the  procuring  activity  thirty  days  after  start  of  system 
engineering  test  and  evaluation,  and  updated  monthly  thereafter.  Failure  reporting 
shall  be  to  the  piece  part  level. 

R.2.5  Reliability  Modeling.  The  contractor  shall  develop  reliability  models  for  all 
system  configurations  in  accordance  with  Task  201  of  MIL-STD-785  and  Task  101 
and  201  of  MIL-STD-756.  The  specific  mission  parameters  and  operational 

constraints  that  must  be  considered  are: _ (or  reference  applicable  SOW  and 

specification  paragraphs). 

R.2.6  Reliability  Allocations.  Reliability  requirements  shall  be  allocated  to  the 
LRU  level  in  accordance  with  Task  202  of  MIL-STD-785. 

R.2.7  Reliability  Prediction.  The  contractor  shall  perform  reliability  predictions 
in  accordance  with  (Task  201  (basic  reliability))  and/or  (Task  20^  (missioh 
reliability))  of  MIL-STD-756.  The  specific  technique  to  be  used  shall  be  method 
2005  parts  stress  analysis  of  MIL-STD-756.  Electronic  part  failure  rates  shall  be 
used  from  MlL-HDBK-217  and  nonelectronic  part  failure  rates  from  RADC-TR-85- 
194.  All  other  sources  of  part  failure  rate  data  shall  require  review  and  ap(;roval  of 

the  procuring  activity  prior  to  use.  A _ environmental  factor,  worst  case 

operating  conditions  and  duty  cycles  shall  be  used  as  a  baseline  for  developing 
part  failure  rates.  The  results  of  the  thermal  analysis  shall  be  included  and  shall 
provide  the  temperature  basis  for  the  predicted  reliability.  The  part  quality  grade 
adjustment  factor  used  shall  be  representative  of  the  quality  of  the  parts  selected 
and  applied  for  this  system  procurement. 

R.2.8  Parts  Program.  The  contractor  shall  establish  and  maintain  a  parts  control 
program  in  accordance  with  Task  207  of  MIL-STD-785  and  Procedure  1  of  MIL- 
STD-965.  Requests  for  use  of  parts  not  on  the  government  generated  and 
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maintained  PPSL  shall  be  submitted  in  accordance  with  the  CDRL.  Amendments 
to  the  PPSL  as  a  result  of  such  requests,  after  procuring  activity  approval,  shall  be 
supplied  to  the  contractor  by  the  Program  Contracting  Officer  not  more  often  than 
once  every  30  days. 

Guidance:  The  level  of  detail  of  the  FMECA  must  be  specified  (e.g.,  part,  circuit 
card,  etc.).  The  closer  the  program  is  to  full  scale  engineering  development,  the 
greater  the  level  of  detail  needed. 

R.2.9  Failure  Modes,  Effects  and  Criticality  Analysis  (FMECA).  The 

contractor  shall  perform  a  limited  FMECA  to  the _ level  to  identify  design 

weaknesses  and  deficiencies.  Potential  failure  modes  shall  be  identified  and 
evaluated  to  determine  their  effects  on  mission  success.  Critical  failures  shall  be 
investigated  to  determine  possible  design  improvements  and  elimination  means. 
MIL-STD-785,  Task  204  shall  be  used  as  a  guide. 

Guidance:  Reliability  critical  items  should  be  required  where  it's  anticipated  that 
the  design  will  make  use  of  custom  VLSI,  hybrids,  microwave  hybrids  and  other 
high  technology  nonstandard  devices.  See  Topic  D5  for  a  critical  item  checklist. 

R.2.10  Reliability  Critical  Items.  Task  number  208  of  MIL-STD-785  applies. 

The  contractor  shall  prepare  a  list  of  critical  items  and  present  this  list  at  all  formal 
reviews.  Critical  items  shall  include;  items  having  limited  operating  life  or  shelf  life, 
items  difficult  to  procure  or  manufacture,  items  with  unsatisfactory  operating 
history,  items  of  new  technology  with  little  reliability  data,  single  source  items,  parts 
exceeding  derating  limits,  and  items  causing  single  points  of  failure. 

R.2.11  Effects  Of  Storage,  Handling,  Transportation.  The  contractor  shall 
analyze  the  effects  of  storage,  handling  and  transportation  on  the  system  reliability. 

R.2.12  Reliability  Qualification  Test.  The  contractor  shall  demonstrate 
compliance  with  the  quantitative  reliability  requirements  in  accordance  with  MIL- 
STD-785  Task  302.  Test  plans  and  reports  shall  be  developed  and  submitted. 

R.2.13  Reliability  Development/Growth  Test.  Test  plans  that  show  data 
tracking  growth,  testing  methods  and  data  collection  procedures  shall  be 
developed  and  submitted  for  the  Growth  Test  Program. 

Guidance:  When  specifying  ESS,  the  level  (circuit  card,  module,  assembly,  etc.) 
at  which  the  screening  is  to  be  performed  must  be  specified.  Different  levels  of 
screening  should  be  performed  at  different  hardware  assembly  levels.  See  R&M 
2000  guidelines  in  Section  T  for  recommended  screening  as  a  function  of 
hardware  assembly  level. 

R.2.14  Environmental  Stress  Screening.  Task  number  301  of  MIL-STD-785 

applies.  A  burn-in  test  of _ (specify  the  number  of  hours  or  temperature  cycles) 

at _ temperature  and _ vibration  level  extremes  shall  be  performed  at  the 

_ level.  At  least _ (hours/cycles)  of  failure  free  operation  shall  be 

experienced  before  termination  of  the  burn-in  test  for  each  unit.  DOD-HDBK-344, 
ESS  of  Electronic  Equipment,  shall  be  used  as  a  guide. 
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Example  Maintainability  Requirements  for  the  System  Specification 


M.1  Maintainability  Requirements 

M.1.1  Maintainability  Quantitative  Requirements.  The  (system  name)  shall  be 
designed  to  achieve  a  mean-corrective-maintenance-time  (Md)  of  no  greater  than 

_ minutes  and  a  maximum-corrective  maintenance-time  (Mmaxct)  of  no  greater 

than _ minutes  {95th  percentile)  at  the  (specify  organization,  intermediate  or 

depot  level),  vrhen  repaired  by  an  Air  Force  maintenance  technician  of  skill  level 
_ or  equivalent. 

Guidance:  Preventive  maintenance  requirements  are  considered  an  option  to  be 
implemented  when  items  are  used  in  the  design  that  are  subject  to  wearout, 
alignment,  adjustment  or  have  fault  tolerance  that  must  be  renewed.  If  the  option  is 
exercised,  then  attach  the  paragraph  below  to  M.1.1. 

M.1.2  Preventive  maintenance  shall  not  exceed _ minutes  for  each  period  and 

the  period  shall  not  be  more  frequent  than  every _ 

M.1.3  The  mean  time  to  restore  system  (MTTRS)  following  a  system  failure  shall 

not  be  greater  than _ MTTRS  includes  all  corrective  maintenance  time  and 

logistics  delay  time. 

M.1.4  The  mean  maintenance  manhours  (M-MMH)  shall  not  be  greater  than _ 

hours  per  year.  M-MMH  is  defined  as  follows:  (operating  hours  per  year)  ^ 

(system  MTBF)  (system  MTTR)  (number  of  maintenance  personnel  required  for 
corrective  action). 

Caution:  Above  definition  of  M-MMH  assumes  that  a  repair  is  made  when  each 
failure  occurs.  If  a  delayed  maintenance  concept  is  anticipated  through  the  use  of 
fault  tolerance,  then  MTBCF  should  be  used  (instead  of  MTBF)  in  the  above 
definition.  If  only  a  limited  number  of  site  visits  are  allowed,  then  this  value  should 
be  used  in  the  above  definition  in  place  of  “operating  hours  per  year  -r  system 
MTBF.” 

M.1.5  Maintainability  Design.  The  system  design  shall  provide  modularity, 
accessibility,  built-in-test  (BIT)  and  other  maintainability  features  to  provide 
installation  simplicity,  ease  of  maintenance  and  the  attainment  of  the 
maintainability  requirements  (both  corrective  and  preventive).  Line  Replaceable 
Units  (LRUs)  such  as  printed  circuit  boards  or  assemblies  shall  be  replaceable 
without  cutting  or  unsoldering  connections.  All  plug-in  modules  shall  be 
mechanically  keyed/coded  to  prevent  insertion  of  a  wrong  module. 

M.1.5.1  Testability.  The  system  design  shall  be  partitioned  based  upon  the  ability 
to  isolate  faults.  Each  item  shall  have  sufficient  test  points  for  the  measurement  or 
stimulus  of  internal  circuit  nodes  to  achieve  the  capability  of  detecting  100  percent 
of  all  permanent  failures  using  full  resources.  Automatic  monitoring  and  diagnostic 
capabilities  shall  be  provided  to  show  the  system  status  (operable,  inoperable, 
degraded)  and  to  detect  90  percent  of  all  permanent  failures.  The  false  alarm  rate 
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due  to  self-test  circuitry  shall  be  less  than  1  percent  of  the  series  failure  rate  Self¬ 
test  circuitry  shall  be  designed  to  correctly  isolate  the  fault  to  a  group  of  four  (4) 
LRUs,  or  less,  95  percent  of  the  time. 

M.1.6  Maintainability  Test  and  Evaiuation.  Maintainability  requirements  for 
the  (system  name)  shall  be  verified  by  the  following; 

M.1.6.1  Maintainability  Anaiysis.  The  results  of  the  final  maintainability 
prediction  shall  be  compared  to  the  quantitative  requirements  and  achievement 
determined  if  the  predicted  parameters  are  less  than  or  equal  to  the  required 
parameters. 

M.1.6.2  Maintainability  Demonstration.  A  maintainability  demonstration  shall 
be  performed  in  accordance  with  Test  Method  (test  method  9  is  commonly 
specified,  see  Appendix  7  for  further  guidance)  of  MIL-STD-471.  A  minimum 
sample  size  of  50  tasks  shall  be  demonstrated.  The  consumer  s  risk  for  the 
maintainability  demonstration  shall  be  equal  to  10  percent.  Fault  detection  and 
isolation  requirements  shall  be  demonstrated  as  part  of  the  maintainability  test. 

M.1.6.3  Testability  Demonstration.  A  testability  demonstration  shall  be 
performed  on  the  (system  name)  in  accordance  with  Notice  2  of  MIL-STD-471  A. 


Example  Maintainability  Requirements  for  the  Statement  of  Work 


M.2  Maintainability  Program  Tasks 

M.2.1  Maintainability  Program.  The  contractor  shall  conduct  a  maintainability 
program  in  accordance  with  MIL-STD-470A  appropriately  tailored  for  full  scale 
development  including  the  following  tasks  as  a  minimum  to  assure  maintainability 
consistent  with  the  requirements. 

M.2.2  Testability  Program.  Testability  characteristics  and  parameters  are 
related  fo,  and  shall  be  treated  as  part  of  the  maintainability  program.  The 
contractor  shall  conduct  a  testability  program  in  accordance  with  MIL-STD-2165 
appropriately  tailored  for  FSD  including  the  following  tasks  as  a  minimum  to  assure 
testability  consistent  with  the  requirements. 

M.2.3  Maintainability  Design  Review.  The  status  of  the  maintainability 
testability  design  shall  be  addressed  at  all  internal  and  external  design  reviews. 

M.2.4  Subcontractor  Control.  The  contractor  shall  specify  maintainability 
requirements  to  all  subcontractors  to  insure  that  (equipment! system  name} 
requirements  of  this  program  are  attained.  Task  102  of  MIL-STD-470A  shall  be 
used  as  a  guide. 

M.2.5  Maintainability/Testability  Modeling.  The  contractor  shall  establish  a 
maintainability  model  using  MiL-STD-470A  Task  201  which  reflects  the 
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construction  and  configuration  of  the  FSD  design.  Linkages  with  MIL-STD-2165 
Task  201  to  relate  testability /diagnostic  design  characteristics  to  maintainability 
parameters  shall  be  provided. 

M.2.6  Maintainability  Prediction.  The  contractor  shall  predict  maintainability 
figures  of  merit  using  Procedure  V  of  MIL-HDBK-472  (Notice  1 )  at  the  on- 
equipment  level.  MIL-STD-470A,  Task  203  shall  be  used  as  a  guide. 

M.2.7  Maintainability/Teatability  Design  Criteria.  The  contractor  shall  develop 
design  criteria  to  be  u^  in  the  design  process  to  achieve  the  specified 
maintainability  and  testability  requirements.  In  addition,  a  design  analysis  showing 
failure  modes,  failure  rates,  ease  of  access,  modularity  and  the  capability  to 
achieve  the  fault  detection/isolation  requirement  shall  be  provided.  RADC-TR-74- 
308  “Maintainability  Engineering  Design  Handbook,”  RADC-TR-82'189  “RADC 
Testability  Notebook,"  Task  202  of  MIL-STD-2165  and  Task  206  of  MIL-STD-470A 
shall  be  used  as  a  guide. 

Guidance:  Maintainability  demonstration  reports  are  only  necessary  if  a 
maintainability  test  is  specified  in  the  maintainability  specification  requirements. 

M.2.8  Maintainability/Testability  Demonstration.  A  Test  plan  and  test  report 
shall  be  submitted  by  the  contractor.  Task  301  of  MIL-STD-470A  and  Task  301  of 
MIL-STD-2165  shall  be  used  as  guides. 
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This  material  has  been  summarized  from  two  excellent  sources  of  information; 

(1 )  “Warranty  Handbook—  A  Reference  Guide  for  Use  by  DoD  Managers  in 
Developing,  Applying,  and  Administering  Warranties”  (Available  from  Defense 
Systems  Management  College,  Fon  Belvoir,  Virginia):  (2)  "Product  Performance 
Agreement  Guide"  (Available  from  AFALC/XRCR  Wright-Patterson  AFB,  Ohio, 
(513)  255-5459.  AV;  785-5459). 

3.1  Warranty  Background 

The  term  “warranty”  is  defined  in  Federal  Acquisition  Regulation  (FAR)  as  “a 
promise  or  affirmation  given  by  a  contractor  to  the  government  regarding  the 
nature,  usefulness,  or  condition  of  the  supplies  or  performance  of  services 
furnished  under  the  contract."  The  terms  “warranty"  and  “guarantee”  are  used 
interchangeably  by  the  Department  of  Defense.  Until  the  passage  of  Public  Law 
98-212  as  part  of  the  Defense  Appropriations  Act  of  1984,  the  use  of  warranties  in 
military  procurements  was  not  mandatory.  However,  warranties  have  frequently 
been  used  by  all  military  services. 

3.2  Military  Warranty  History 

3.2.1  1960  to  1980 

In  1964,  Section  1-324  of  the  Armed  Services  Procurement  Regulation  (ASPR) 
was  issued,  containing  regulations  on  the  use  of  warranties.  The  section,  which 
has  been  updated  periodically,  has  been  generally  interpreted  to  mean  that  use  of 
an  extensive,  long-term  warranty  should  be  the  exception  rather  than  the  rule.  For 
commercial  items,  the  military  normally  obtains  a  standard  warranty  if  the  planned 
usage  of  the  item  is  consistent  with  normal  usage.  Early  government  controls 
against  acquiring  defective  material  included  warranty  control  against  latent 
defects.  In  the  late  1960s  and  early  1970s.  more  extensive  warranty  forms  were 
tried,  such  as  on  the  Navy  F-4  gyro  (failure-free  warranty  (FFW).  and  the  Air  Force 
ARN-1 18  TACAN  (reliability  improvement  warranty  (RIW)).  The  services  supported 
research  studies  to  evaluate  warranty  applications  and  to  develop  analysis  and 
implementation  tools.  After  evaluating  a  number  of  early  warranty  problems, 
researchers  concluded  that  a  properly  structured  and  implemented  military 
warranty  can  offer  significant  potential  for  achieving  desired  operational 
performance  at  reasonable  cost. 

3.2.2  Warranty  Initiatives  in  the  1980s 

The  successful  use  of  such  warranty  forms  as  MTBF  guarantees  and  RIW  during 
the  1970s  provided  a  basis  for  extending  warranty  applications  to  a  broader  class 
of  programs.  In  1980,  the  Air  Force  issued  the  first  Product  Performance 
Agreement  Guide,  which  provided  a  summary  of  the  features  of  various  forms  of 
warranties  that  could  be  used  in  military  procurements.  In  1982,  the  Product 
Performance  Agreement  Center  (PPAC)  was  established  to  provide  a  focal  point 
for  Air  Force  use  of  product  performance  agreements  and  warranties  (the  term 
“product  performance  agreement”  was  adopted  by  the  Air  Force  to  mean  all  forms 
of  contractural  agreements  relating  to  the  performance  of  a  product).  Also  in  1982, 
the  Department  of  Defense  issued  a  set  of  initiatives,  which  became  known  as  the 
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Carlucci  Initiatives,  to  improve  and  streamline  the  acquisition  process.  They 
included  warranties  as  one  means  of  achieving  desired  levels  of  system  reliability 
and  maintainability.  Congressional  interest  in  warranty  as  a  means  of  ensuring 
acceptable  field  performance  started  with  the  passage  of  Public  Law  98-212,  which 
was  part  of  the  1984  Defense  Appropriations  Act,  mandating  that  warranties  be 
included  in  the  production  contract. 

3.3  Current  Warranty  Law 

The  Defense  Procurement  Reform  Act  (Public  Law  98-525),  effective  January 
1985,  established  Title  10,  Section  2403,  of  the  United  States  Code,  entitled  "Major 
Weapon  Systems:  Contractor  Guarantees."  The  law  requires  that  the  prime 
contractor  for  a  production  weapon  system  provide  written  guarantees,  starting 
with  procurements  after  1  January  1985.  Table  3.1  summarizes  the  essential 
features  of  the  law. 


Table  3.1 ;  Summary  of  1985  Warranty  Law 


Factor  Definition  Deacription 


Coverage 

Weapon  Systems 

Used  in  combat  missions;  unit  cost  is  greater 
than  $100,000.  or  total  procurement  exceeds 
$10,000,000 

Warrantor 

Prime  contractor 

Parly  that  enters  into  direct  agreement  with 
US  to  furnish  part  or  all  of  weapon  system 

Warranties 

Design  and  manufacturing 
requirements 

Item  meets  structural  and  engineering  plans 
.nnd  manufacturing  particulais. 

Defects  in  materials  and 
workmanship 

Item  IS  free  from  such  defects  at  the  time  it  is 
delivered  to  the  government. 

Essential  performance 
requirements 

Operating  capabilities  or  maintenance  and 
reliability  characteristics  of  item  are 
necessary  for  fulfilling  the  military 
requirements 

Exclusions 

GFRGFE.GFfVI 

Items  provided  to  the  contractor  by  the 
government 

Waivers 

Necessary  m  the  interest  of 
national  defense,  warranty  not 
cost-effective 

Assistant  Secretary  of  Defense  or  Assistant 
Secretary  of  the  Military  Department  is 
lowest  authority  for  granting  waiver  prior 
notification  to  House  and  Senate 
committees  required  tor  ma)or  weapon 
system 

3.4  US  Air  Force  Warranties 

Air  Force  policy  documents  indicate  that  the  Air  Force  will  require  a  warranty  plan 
for  each  procurement  documenting  the  responsibilities,  decisions,  taskings  and 
strategies  for  warranties.  Table  3.2  lists  offices  that  have  been  designated  Air 
Force  warranty  focal  points. 
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Table  3.2:  Air  Force  Warranty  Focal  Points 


0<1ic« 

AdOrMS 

T*l«f>hon«  Number 

Warranty  Contracting 

HQ  USAF/RDCS 

Pentagon 

Washirrgton  DC  20330 

(202)  697-6400 

Warranty  Administration 

HQ  USAF/LEYE 

Pentagon 

Washirrgton  DC  20330 

(202)697-0311 

Air  Force  Systems  Command 

HQ  AFSC/PLE 

Andrews  AF8  DC  20334 

(301)981-4076 

Air  Force  Logistics  Command 

HQ  AFLC/MMA 

Wright-Patterson  AF6  OH  45433 

(513)257-7119 

Warranty  Data  Base  and 
Consulting 

Product  Performance 

Agreement  Center 
Wright-Patterson  AFB  OH  45433 

(513)255-5459 

3.5  Product  Performance  Agreement  Center  (PPAC) 

The  Air  Force  PPAC  was  established  in  1982  to  assist  Air  Force  activities  involved 
in  the  acquisition  of  defense  systems  and  their  components  in  selecting, 
structuring,  pricing,  negotiating  and  implementing  effective  Product  Performance 
Agreements  (PPAs)  and  related  business  arrangements.  To  promote  the  use  of 
PPAs  in  Air  Force  procurements  pursuant  to  10  DSC  2403,  and  to  promote 
effective  application  and  management  of  PPAs  at  all  levels,  the  PPAC  currently 
performs  the  following  functions: 

•  Serves  as  the  central  repository  of  Air  Force  PPA-related  data 

•  Analyzes  the  effectiveness  of  existing  and  proposed  PPAs 

•  Develops  improved  contract  clauses  and  related  concepts 

•  Provides  technical  assistance  to  Air  Force  activities  in  selecting,  tailoring, 
pricing,  negotiating  and  administering  appropriate  agreements 

•  Formulates  proposed  policy  guidance  for  HQ  USAF  consideration  concerning 
application  of  PPAs  to  Air  Force  acquisitions. 

3.6  Warranty  Classifications 

A  number  of  warranty  classification  schemes  have  been  developed  to  describe 
alternatives  available  to  procurement  activities.  The  usual  classification  scheme 
distinguishes  between  assurance  and  incentive  forms  of  warranties.  Table  3  3 
compares  the  characteristics  of  these  two  types. 
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Table  3.3:  Comparison  of  Assurance  and  Incentive 

Types  of  Warranties 

Factor 

Asauranca  Warranty 

Incentive  Warranty 

Basic  Intent 

Meet  minimum  performance  and 
R&M  levels. 

Exceed  minimum  levels. 

Warranty  Price 

Expected  to  be  minimal,  from  0  up 
to  1  or  2  percent  per  year  of 
hardware  price 

May  be  significant,  up  lo  7  or  8 
percent  per  year  of  hardware 
price 

Warranty  Duration 

Limited — generally  1  year  or  less. 

Can  be  extensive — 3  or  more 
years. 

Technology  Factors 

Warranted  item  is  well  within 
state-of-the-art  (SOA),  or  SOA  is 
so  severely  "pushed "  that  only 
limited  warranty  protection  is 
realistic 

Wa  J  item  pushes  SOA.  so 

there  is  need  to  protect  against 
failure  and  there  is  opportunity  for 
growth. 

Contractor 

Contractor  has  limited  opportunity 
to  control  and  improve 
performance  prior  to  and  during 
warranty. 

Contractor  has  significiint 
opportunity  to  control  and  improve 
performance. 

Competition 

Should  not  reduce  future 
competitive  climate. 

May  significantly  reduce 
competitive  climate. 

Administration 

Generally  not  a  severe  burden. 

May  require  complex  procedures. 

3.7  Product  Performance  Agreement  Guide  Warranties 

In  November  1985,  The  Air  Force  Product  Performance  Agreement  Guide  listed  28 
forms  of  PPAs  and  provided  additional  background  and  guidance  on  their  potential 
application.  The  four  most  commonly  used  incentive  forms  of  warranties  are  listed 
in  Table  3.4. 
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3.8  Contractor  Roliability  Motivations 

Reliability  is  one  of  the  principal  system  performance  parameters  that  the  warranty 
law  addresses.  Reliability  differs  from  quality  in  the  sense  that  it  pertains  to  the 
long-term  performance  of  the  system — or.  the  mean  time  between  system  failures. 

Contractors  generally  have  a  positive  attitude  toward  quality.  Quality  audits  are 
normally  performed  on  all  submitted  products  and  rejections  result  in  adued 
expense  and  reduced  profit.  Reliability,  on  the  other  hand,  is  more  elusive:  it 
cannot  be  measured  easily,  and,  in  some  respects,  it  does  not  offer  immediate, 
positive  motivations  to  a  contractor,  in  fact,  one  can  argue,  perhaps  cynically,  that 
without  a  warranty,  failures  of  a  deployed  system  mean  more  profit  to  a  contractor  if 
the  contractor  is  providing  maintenance  or  spares.  In  addition,  if  reliability  is  a 
serious  problem,  the  same  contractor  is  probably  tasked  to  develop  a  fix  and  to 
retrofit  existing  systems.  Figure  3.1  illustrates  contractor  profit  motivation  with 
a  warranty. 


Figure  3.1 :  Contractor  Profit  Motivation — Warranty 


3.9  Other  Warranty  Motivations 

There  are  other  motivations,  besides  reliability,  that  can  be  associated  with  a 
warranty.  The  warranty  commitment  forces  the  contractor  to  think  seriously  beyond 
just  having  the  product  accepted.  Being  involved  throughout  the  warranty  period 
may  cause  the  contractor  to  be  concerned  with  maintenance,  diagnostics,  training, 
data  and  other  logistics  and  support  factors.  As  an  example,  warranties  have  been 
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written  under  which  the  contractor  is  not  reimbursed  for  processing  good  units 
returned  unless  the  percentage  of  such  returns  is  very  high.  Since  such  processing 
is  costly,  the  contractor  may  be  motivated  to  improve  the  built-in-test  equipment, 
technical  manuals,  test  equipment,  and  other  elements  associated  with  failure 
detection  and  verification. 

Another  motivational  factor  concerns  maintenance  efficiency.  If  the  contractor  has 
to  repair  all  warranted  failures,  it  is  important  that  there  be  an  efficient  and  effective 
repair  process.  There  have  been  a  number  of  instances  in  which  such  warranties 
influenced  the  contractor  to  design  for  maintenance  as  well  as  reliability. 

When  the  contractor  views  warranty  as  a  potential  profit  source  and  a  means  for 
achieving  a  competitive  edge,  a  number  of  positive  motivational  factors  may  be 
present.  Producers  of  quality  equipment  need  not  add  significant  warranty 
contingency  or  risk  funds  to  their  price  to  cover  future  failures,  and  they  need  not 
spend  all  of  their  warranty  funds  to  fix  a  poor  product.  A  warranty  environment 
encourages  producers  to  achieve  and  maintain  a  quality  product. 

3.10  Summaries  of  Early  Programs 

Reviews  of  a  number  of  warranty  programs  begun  during  the  1970s  have  generally 
concluded  that  long-term  warranties  can  provide  significant  improvements  m 
operational  performance  (R&M). 

Data  collected  by  ARINC  Research  Corporation  compared  field  MTBF  values 
under  a  warranty  program  with  goal  values,  some  of  which  were  contractually 
guaranteed.  The  results  show  that  the  field  reliability  exceeded  the  goal  value  for 
all  the  programs  tested  but  one.  The  data  were  developed  over  a  period  when  field 
reliability  for  unwarranted  systems  often  was  much  lower  than  was  specified, 
predicted  or  tested.  Although  the  results  suggested  that  warranty  programs 
provide  a  mechanism  for  achieving  reliable  equipment,  there  are  several  factors 
to  consider; 

•  The  data  represent  programs  that  were  carefully  selected  for  warranty 
application. 

•  Many  of  the  warranty  programs  entailed  fairly  extensive  forms  of  warranty,  with  a 
great  deal  of  effort  given  to  structuring  the  terms  and  conditions  properly. 

•  Advertising  that  an  equipment  is  to  be  warranted  is  one  way  to  help  ensure  that 
contractor-proposed  MTBF  values  are  realistic.  While  this  is  beneficial,  it  can 
lead  to  misinterpretation  of  results  when  data  from  warranted  equipment  are 
compared  with  similar  data  from  nonwarranted  equipment.  Without  an  expected 
warranty  commitment,  expected  MTBF  values  often  became  inflated. 

3.1 1  Summaries  of  Recent  Programs 

Since  passage  of  the  1984  law,  hundreds  of  procurements  have  been  contracted: 
presumably,  most  comply  with  the  statutory  regulation.  Unfortunately,  there  is  very 
little  field  experience  to  assess  the  workability  and  effectiveness  of  the  warranty 
provisions.  The  lack  of  relevant  data  makes  evaluation  of  warranties  difficult.  A 
recent  Air  Force  PPAC  study  assessing  the  effectiveness  of  Air  Force  RIW 
programs  provided  some  findings  on  the  data  problems. 
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•  Administration  and  tracking  of  warranted  items  vary  from  program  to  program. 

•  Warranty  programs  are  diverse  in  the  amount  of  reporting  and  relevant  data 
available  to  accomplish  an  adequate  assessment. 

•  Most  warranty  programs  do  not  undergo  the  pre-RIW  analyses  (trade  studies 
and  cost-benefit  analyses)  necessary  to  determine  the  best  way  to  apply 

a  warranty. 

In  light  of  these  findings,  the  reported  effectiveness  of  warranties  should  be  viewed 
with  caution,  and  it  should  be  recognized  that  the  documentation  and  collection  of 
warranty  data  has  not  been  uniform,  thereby  impeding  direct  comparison  and 
conclusions. 
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This  Appendix  contains  an  example  set  of  design  guidelines  structured  to  include 
verification  methods.  These  guidelines  are  an  example  only  and  don't  apply  to 
all  situations. 

a.  Thermal  Design 

(1 )  Integrated  Circuit  Junction  Temperatures. 

Design  Guideline:  Thedcjiyn  u.’  iiie  envircnnienta:  cooling  system  (ECS)  should 
be  capable  of  maintaining  an  average  integrated  circuit  junction  temperature  of 
55°C  or  less  under  typical  operating  conditions.  Under  worst  case  steady  state 
conditions,  components  should  operate  at  least  50°C  below  their  rated  maximum 
junction  temperature. 

Analysis  Recommendation:  Thermal  finite  element  analysis  should  be 
performed  to  project  operating  temperatures  under  specified  environmental 
conditions.  The  analysis  should  consider  ECS  performance,  environmental 
impacts,  and  system  thermal  design.  Average  junction  temperatures  should 
include  all  integrated  circuits  within  the  system.  Average  temperature  rise  should 
include  all  components  on  an  individual  module. 

Test  Recommendation:  Thermally  instrumented  observations  should  be  made  of 
components  under  specified  environmental  conditions.  Instrumentation  can  be  by 
direct  contact  measurement  or  by  infrared  photography. 

(2)  Thermal  Gradients. 

Design  Guideline:  The  maximum  allowable  temperature  rise  from  any  junction  to 
the  nearest  heat  sink  should  be  25°C.  The  average  temperature  rise  from 
integrated  circuit  junctions  to  the  heat  sink  should  be  no  greater  than  IS'C.  To 
minimize  gradients,  more  complex  and  power-intensive  devices  should  be  placed 
to  minimize  their  operating  temperature. 

Analysis  Recommendation:  Automated  design  tools  that  perform  component 
placement  should  be  programmed  to  produce  this  result.  A  thermal  finite  element 
analysis  should  be  used  to  evaluate  the  projected  thermal  gradient  under  the 
specified  environmental  conditions. 

Test  Recommendation:  Thermally  instrumented  observation  of  components 
under  specified  environmental  conditions.  Instrumentation  can  be  by  direct  contact 
measurement  or  by  infrared  photography. 

(3)  Thermal  Expansion  Characteristics. 

Design  Guideline:  Component  and  board  materials  should  be  selected  with 
compatible  thermal  coefficients  of  expansion  (TCE).  Additionally,  coldplate 
materials  should  be  selected  for  TCE  compatibility  with  the  attached  printed  wiring 
board.  TCE  mismatch  results  in  warpage  of  the  laminated  assembly,  which  can 
reduce  module  clearances  and  stress  PWB  component  leads  and  solder  joints. 

Analysis  Recommendation:  A  finite  element  analysis  should  be  performed  to 
identify  the  stress  patterns  in  the  solder  joints  attaching  the  components  to  the 
board.  TCE  compatibility  should  be  evaluated  for  the  components,  PWB, 
and  coldplate. 
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Test  Recommendation:  Environmental  stress  tests  should  be  utilized  in  the 
development  phase  to  verify  the  design  analysis,  and  environmental  stress 
screening  should  be  used  in  production  to  ensure  consistency  throughout  the 
production  cycle. 

(4)  Heat  Transport  Media. 

'design  G'llHeiine:  The  design  should  use  a  thermal  corduotion  medium  that  is 
int^ral  to  the  mechanical  design  of  the  board  or  module.  Heat  pipes,  metal  rails, 
or  internal  planes  are  examples  of  thermally  conductive  media.  The  unit  should 
meet  temperature  design  requirements  by  cooling  through  the  integral  thermal 
conduction  medium  without  depending  on  any  other  heat  loss. 

Analysis  Recommendation:  Thermal  finite  element  analysis  should  be  used  to 
project  heat  flow  under  specified  environmental  conditions.  Modules  employing 
heat  pipes  for  cooling  should  meet  operating  temperature  requirements  when  the 
module  heat  sink  is  inclined  at  an  angle  of  90  degrees  from  the  horizontal. 

Test  Recommendation:  Thermally  instrumented  observation  should  be  made  of 
components  under  specified  environmental  conditions.  Instrumentation  can  be  by 
direct  contact  measurement  or  by  infrared  photography. 

(5)  Component  Attachment. 

Design  Guideline:  Surface  contact  should  be  maximized  between  the 
component  and  the  heat  transport  media.  This  can  be  achieved  by  direct  pressure, 
thermal  compounds,  or  solder.  The  technique  used  should  be  reversible  for 
component  removal  during  board  repairs  such  that  damage  is  not  induced  to 
nearby  devices.  If  a  thermal  compound  is  used,  it  should  not  migrate  or  react  with 
other  components  during  testing  or  service  use. 

Analysis  Recommendation:  Specialized  stress  analyses  should  be  performed 
to  quantify  thermal  and  mechanical  stresses  involved  in  removing  the  component 
from  the  ^ard  after  production  installation. 

Test  Recommendation:  Demonstration  of  repair  techniques  should  be  performed 
early  in  the  development  phase. 

(6)  Thermal  Cycling. 

Design  Guideline:  The  unit  should  be  designed  to  dampen  its  thermal  response 
to  the  thermal  excursions  required  by  the  specification.  This  can  be  achieved  by 
using  a  large  thermal  mass  or  by  using  the  cooling  medium  to  insulate  the  unit  from 
its  environment  to  the  maximum  extent  possible. 

Analysis  Recommendation:  Thermal  finite  element  analysis  to  project  heat  flow 
and  temperature  excursions  under  specified  environmental  conditions. 

Test  Recommendation:  Thermally  instrumented  observation  of  components 
under  specified  environmental  excursions.  Instrumentation  can  be  by  direct 
contact  measurement  or  by  infrared  photography. 
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b.  Testability  D«sign 

(1)  Bottom-up  Fault  Reporting. 

Design  Guideline:  incorporate  autonomous  self-testing  at  the  lowest  levels  that 
are  technically  feasible.  Utilize  positive  indication  to  report  chip,  module  and 
subsystem  status.  The  design  should  not  depend  upon  external  stimuli  to  perform 
fault  detection  or  isolation  to  a  replaceable  element. 

Analysis  Recommendation:  As  soon  as  automated  testability  analysis  tools 
become  available,  they  should  be  used  for  the  applicable  engineering  design 
workstations. 

Test  Recommendation:  Hardware  demonstration  should  be  conducted  early  in 
the  development  phase  to  verify  simulation  results  through  the  insertion  of  faults 
using  the  currently  available  version  of  the  operational  program,  firmware,  and 
microcode. 

(2)  Fault  Logging. 

Design  Guideline:  Modules  should  contain  a  non-volatile  fault  log  that  can  be 
accessed  by  a  system  maintenance  controller  or  by  test  equipment.  The  use  of  the 
fault  log  will  improve  reliability  by  reducing  depot  “Cannot  Duplicates.”  Failure  of 
the  fault  log  should  not  cause  a  critical  system  failure,  but  should  be  observable  to 
the  maintenance  controller. 

Analysis  Recommendation:  Compliance  should  be  verified  by  inspection. 
Operation  should  be  verified  by  simulation. 

Test  Recommendation:  Not  applicable. 

(3)  Start-up  Built-In-Test  (BIT) 

Design  Guideline:  The  module  should  execute  a  BIT  internal  diagnostic  routine 
immediately  after  power-up  or  receipt  of  an  “Execute  BIT”  command.  BIT  should 
provide  a  complete  functional  test  of  the  module  to  the  maximum  extent  possible 
without  transmitting  any  signals  on  external  interface  media.  BIT  should  provide  a 
complete  functional  test  of  the  module  and  should  include; 

(1 )  Verification  of  internal  data  paths. 

(2)  Verify  station  physical  address. 

(3)  Verity  message  identification  process  from  system. 

(4)  Verify  proper  functioning  of  all  internal  memory  and  other  components. 

Any  failures  encountered  during  execution  of  BIT  should  be  retried  at  least  once  to 
confirm  the  response.  Any  confirmed  failures  should  prevent  the  module  from 
becoming  enabled.  A  failed  module  should  respond  only  to  “RESET,”  “Execute 
BIT"  and  "Report  Status”  commands. 

Analysis  Recommendation:  System  design  simulation  tools  should  be  used  to 
verify  operation  of  the  BIT.  These  tools  should  include  fault  simulations  as  well  as 
operational  simulation. 
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Test  Recommendation:  Hardware  demonstration  should  be  conducted  early  in 
the  development  phase  to  verify  simulation  results  through  insertion  of  faults  using 
currently  available  versions  of  the  operational  program,  firmware,  and  microcode. 

(4)  Background  Diagnostics. 

Design  Guideline:  During  normal  operation,  the  module  should  continuously 
monitor  itself  through  a  background  diagnostic  test.  The  background  diagnostic 
should  provide  coverage  to  the  maximum  extent  possible  without  interfering  with 
normal  station  operation.  Failure  of  any  test  in  the  background  diagnostic  should 
cause  the  module  to  re-execute  the  failed  test  to  screen  out  transient  anomalous 
responses.  If  the  failure  is  confirmed,  the  module  should  become  immediately 
disabled. 

Analysis  Recommendaiion:  System  design  simulation  tools  should  be  used  to 
verify  operation  of  the  BIT.  These  tools  should  include  fault  simulations  as  well  as 
operational  simulation. 

Test  Recommendation:  Hardware  demonstration  should  be  conducted  early  in 
the  development  phase  to  verify  simulation  results  through  insertion  o[  faults  using 
currently  available  versions  of  the  operational  program,  firmware,  and  microcode. 
Hardware  demonstration  may  be  performed  by  physically  inserting  faults  in  a 
module  or  by  instrumenting  a  module  to  allow  insertion  of  faults  through  external 
methods. 

c.  Mechanical  Packaging  Design 

(1)  Mechanical  Insertion/Extraction-Induced  Stresses. 

Design  Guideline:  Each  module  should  withstand,  without  damage  or 
separation,  a  minimum  force  equal  to  at  least  100  pounds  on  insertion  and  four 
ounces  per  contact  on  extraction.  Additionally,  the  backplane  for  the  assembly 
should  withstand  the  same  forces  at  all  module  positions  applied  repeatedly  in  any 
sequence  with  any  combination  of  modules  present  or  missing. 

Analysis  Recommendation:  A  mechanical  loads  analysis  should  be  performed 
to  verify  compliance  with  the  mechanical  requirements. 

Test  Recommendation:  The  total  computed  force  should  be  applied  to  simulate 
module  insertion  and  extraction.  The  force  should  be  applied  in  2  seconds  and 
maintained  for  15  seconds. 

(2)  Insertion/Extraction  Durability. 

Design  Guideline:  Modules  should  be  capable  of  withstanding  500  cycles  of 
mating  and  unmating  with  no  degradation  of  module  performance.  The  module 
should  also  be  capable  of  withstanding  500  cycles  of  lateral  displacement  to 
simulate  the  use  of  thermal  clamping  devices.  The  backplane  of  the  module’s  host 
assembly  should  be  capable  of  withstanding  500  of  the  same  cycles  on  each  of  its 
module  positions. 

Analysis  Recommendation:  A  mechanical  loads  analysis  should  be  performed 
to  verify  compliance  with  the  mechanical .  -jquirements. 
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Test  Recommendation:  Each  module/backplane  position  should  be  subject*^d  to 
500  cycles  of  insertion/extraction.  The  maximum  specified  insertion  and  extraction 
forces  should  be  applied  in  2  seconds  and  maintained  for  15  seconds.  Five 
hundred  lateral  displacement  cycles  should  be  applied  to  the  module. 


(3)  Mechanical  Vibration-Induced  Stresses. 

Design  Guideline:  The  larger  components  are  more  susceptible  to  mechan  cal 
stresses  because  they  have  a  larger  mass  and  because  they  are  more  consti  lined 
by  the  high  number  of  pin-outs  that  act  as  attachment  points.  Module  stiffnes*. 
should  be  maximized  to  prevent  board  flexing  resulting  in  stress  fractures  at  the 
solder  joints  or  component  leadframe. 

Analysis  Recommendation:  Mechanical  finite  element  analysis  should  be 
performed  to  identify  module  characteristics  throughout  the  specified  vibrational 
environment. 

Test  Recommendation:  Developmental  units  should  be  specially  instrumented 
with  accelerometers  early  in  the  development  program.  These  units  could  use 
dummy  masses  attached  using  the  intended  production  technique.  Standard 
endurance  and  qualification  tests  should  be  performed  in  accordance  with  MIL- 
STD-810,  "Environmental  Test  Methods  and  Engineering  Guidelines.” 


(4)  Module  Torque  Stresses. 

Design  Guideline:  The  module  should  be  capable  of  withstanding  a  6  inch-pound 
torque  applied  in  2  seconds  and  maintained  for  15  seconds  in  both  directions  along 
the  header  in  a  direction  perpendicular  to  the  plane  of  the  header  without 
detrimental  effect  to  the  mechanical  or  electrical  properties  of  the  module. 

Analysis  Recommendation:  A  mechanical  loads  analysis  should  be  performed 
to  verify  compliance  with  the  mechanical  requirements. 

Test  Recommendation:  The  required  torque  should  be  applied  in  2  seconds  and 
maintained  for  15  seconds.  During  the  time  the  torque  is  applied,  the  module 
should  be  rigidly  supported  within  a  zone  between  the  interface  plane  and  0.5  inch 
above  the  interface  panel. 


(5)  Module  Cantilever  Load. 

Design  Guideline:  The  module  should  be  capable  of  withstanding  a  force  of  2 
pounds  applied  perpendicular  to  the  header  height  along  the  center  line  midway 
between  the  two  extractor  holes. 

Analysis  Recommendation:  A  mechanical  loads  analysis  should  be  performed 
to  verify  compliance  with  the  mechanical  requirements. 

Test  Recommendation:  The  required  force  should  be  applied  in  two  directions 
and  should  be  applied  in  2  to  10  seconds  and  maintained  for  10  to  15  seconds 
without  detrimental  effect  to  the  header  structure. 
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(6)  Module  Retention. 

Design  Guideline:  Module  retention  techniques  must  be  carefully  designed  to 
integrate  the  insertion  mechanism,  required  connector  insertion  force,  thermal 
contact  area,  and  extraction  mechanism.  Conventional  elect'-onics  have  required 
the  same  considerations,  but  to  a  lesser  degree  because  of  their  more 
conventional  housings. 

Analysis  Recommendation:  Specialized  analyses  should  be  used  to  quantify 
torque  requirements  and  limitations  of  the  wedge-clamping  device,  lever  moments 
of  insertion  or  extraction  devices,  tolerance  buildups  of  the  module  slot  and 
connector  placement,  and  mechanical  deflections  of  the  backplane. 

Test  Recommendations:  Standard  endurance  and  qualification  tests  in 
accordance  with  MIL-STD-810,  “Environmental  Test  Method?  and  Engineering 
Guidelines." 

(7)  Connector  Contact  Integrity. 

Design  Guideline:  Each  contact  pin,  as  mounted  in  the  connector,  should 
withstand  a  minimum  axial  force  of  20  ounces. 

Analysis  Recommendation:  A  mechanical  loads  analysis  should  be  performed 
to  verify  compliance  with  the  mechanical  requirements. 

Test  Recommendation:  The  required  force  should  be  applied  in  2  secon.is  along 
the  length  of  the  contact  in  either  direction  and  maintained  for  15  seconds 

(8)  Connector  Float. 

Design  Guideline:  The  connector-to-module  interface  should  be  sufficiently 
flexible  to  compensate  for  specified  misalignments  or  tolerance  buildup  between 
the  module  and  the  backplane  connector  shells. 

Analysis  Recommendation:  Tolerance  review  should  be  performed  early  in 
design  process. 

Test  Recommendation:  Demonstration  testing  can  be  nerformed  easily  during 
the  initial  mechanical  design  phase. 

(9)  Keying  Pin  integrity. 

Design  Guideline:  When  installed  in  the  module,  the  keying  pins  should  meet  the 
following  integrity  requirements.  Each  keying  pin  should  withstand  a: 

•  torque  of  20  inch-ounces 

•  pullout  force  of  9  pounds 

•  pushout  force  of  40  pounds 

•  cantilever  load  of  10  pounds 

Analysis  Recommendation:  A  mechanical  loads  analysis  should  be  performed 
to  verify  compliance  with  the  mechanical  requirements. 


A-36 


RADC  RELIABILITY  ENGINEER  .  TOOLKIT 


EXAMPLE  DESIGN  GUIDELINES 


Test  Recommendation:  The  required  forces  should  be  applied  to  the  keying  pin 
in  2  seconds  and  maintained  for  15  seconds. 

d.  Power  Supply  Design 

(1)  Overcurrent  Protection: 

Design  Guideline:  The  power  supply  should  supply  125  percent  of  its  rated 
output  for  2  ±  0.25  seconds,  after  which  the  power  supply  will  shut  down  (shut 
down  is  defined  as  all  outputs  at  less  than  1  mv  and  1  ma  current,  but  all  status  and 
control  lines  still  operating).  Operation  should  not  resume  until  the  power  supply  is 
reset.  In  addition,  the  power  supply  outputs  should  be  short  circuit  protected. 

Analysis  Recommendation:  Compliance  with  the  specified  operation  should  be 
verified  throughout  the  design  process. 

Test  Recommendation:  Specified  operation  of  the  protective  device  should  be 
induced  by  application  of  the  anomalous  condition  protected  against.  Correct 
operation  of  the  protective  device  should  be  observed.  Normal  specified  power 
supply  operation  should  be  verified  after  removal  of  the  anomalous  condition. 

(2)  Overvoltage  Protection. 

Design  Guideline:  The  output  should  be  sensed  for  overvoltage.  An  overvoltage 
on  the  output  should  immediately  shut  down  the  power  supply.  Operation  should 
not  resume  until  the  power  supply  is  reset.  The  overvoltage  limits  should  be 
compatible  with  device  logic  absolute  maximum  limits.  The  overvoltage  protection 
and  sense  circuits  should  be  constructed  such  that  an  overvoltage  on  a  failed 
power  supply  will  not  cause  any  other  paralleled  power  supply  to  also  shut  down. 

Analysis  Recommendation:  Compliance  with  the  specified  operation  should  be 
verified  throughout  the  design  process. 

Test  Recommendation:  Specified  operation  of  the  protective  device  should  be 
induced  by  application  of  the  anomalous  condition  protected  against.  Correct 
operation  of  the  protective  device  should  be  observed.  Normal  specified  power 
supply  operation  should  be  verified  after  removal  of  the  anomalous  condition. 

(3)  Abnormal  Thermal  Operation. 

Design  Guideline:  In  the  event  of  an  above-normal  internal  temperature,  the 
power  supply  should  be  capable  of  continued  operation  at  a  reduced  power  output. 
Thermal  sense  circuits  should  regulate  the  output  to  the  extent  necessary  to  keep 
semiconductor  junctions  at  or  below  specified  levels.  The  power  supply  should 
resume  operation  at  rated  output  if  internal  temperatures  return  to  normal. 

Analysis  Recommendation:  Compliance  with  the  specified  operation  should  be 
verified  throughout  the  design  process. 

Test  Recommendation:  Specified  operation  of  the  protective  device  should  be 
induced  by  application  of  the  anomalous  condition  protected  against.  Correct 
operation  of  the  protective  device  should  be  observed.  Normal  specified  power 
supply  operation  should  be  verified  after  removal  of  the  anomalous  condition. 
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(4)  Thermal  Shutdown. 

Design  Guideline;  When  thermal  limiting  is  no  longer  capable  of  maintaining 
internal  temperature  at  an  acceptable  level,  the  power  supply  should  automatically 
shut  down.  Operation  should  not  resume  until  the  power  supply  is  reset. 
Temperature  sense  circuits  should  remain  active  duhog  shut  down. 

Analysis  Recommendation:  Compliance  with  the  specified  operation  should  be 
verified  throughout  the  design  process. 

Test  Recommendation:  Specified  operation  of  the  protective  device  should  be 
induced  by  application  of  the  anomalous  condition  protected  against.  Correct 
operation  of  the  protective  device  should  be  observed.  Normal  specified  power 
supply  operation  should  be  verified  after  removal  of  the  anomalous  condition. 

(5)  Power  Supply  Status  Reporting. 

Design  Guideline:  There  should  be  an  interface  on  each  power  supply  module 
that  will  allow  data  communication  between  the  power  supply  and  a  CPU  located 
on  a  separate  module.  Each  power  supply  module  will  be  addressee  individually. 
The  data  and  control  lines  should  interface  to  the  power  supply  module  through  the 
backplane  connector.  The  following  power  supply  parameters  should  be  read  by 
the  CPU: 

•  overcurrent  status 

•  overvoltage  status 

•  thermal  limiting  mode  status 

•  thermal  shutdown  status 

•  percentage  of  full  output  power  available 

The  following  commands  should  be  issued  by  the  CPU  to  the  power  supply 
module: 

•  reset 

•  percentage  of  full  output  power  required 

Analysis  Recommendation:  Compliance  with  the  specified  operation  should  be 
verified  throughout  the  design  process. 

Test  Recommendation:  Specified  operation  of  the  protective  device  (i.e.. 
monitoring  mechanism  and  control)  should  be  induced  by  application  of  the 
anomalous  condition  protected  against.  Correct  operation  of  the  protective  device 
should  be  observed.  Norma!  specified  power  supply  operation  should  be  verified 
after  removal  of  the  anomalous  condition. 

(6)  Power  Supply  Input  Protection. 

Design  Guideline:  The  power  supply  should  automatically  shut  down  if  the  input 
voltage  is  not  within  the  specified  allowable  range,  and  at  any  time  when  the 
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Control  circuits  in  the  power  supply  do  not  have  adequate  voltage  to  regulate  the 
outputs.  This  should  include  the  time  during  normal  start-up  when  generators  are 
not  producing  their  normal  output  voltage. 

Analysis  Recommendation:  Compliance  with  the  specified  operation  should  be 
verified  throughout  the  design  process. 

Test  Recommendation:  Specified  operation  of  the  protective  device  should  be 
induced  by  application  of  the  anomalous  condition  protected  against.  Correct 
operation  of  the  protective  device  should  be  observed.  Normal  specified  power 
supply  operation  should  be  verified  after  removal  of  the  anomalous  conoition. 

(7)  Backplace  Conditions. 

Design  Guideline:  A  sufficient  number  of  connector  pins  should  be  paralleled  so 
that  no  backplane  connector  pin  carries  more  than  5  amps  of  current. 

Analysis  Recommendation:  Compliance  wjth  the  specified  operation  should  be 
verified  throughout  the  design  process. 

Test  Recommendation:  Not  applicable. 

(8)  M-of-N  Power  Supply  Redundancy. 

Design  Guideline:  The  quantity  of  power  supplies  for  a  system  of  functional 
elements  should  be  determined  to  allow  uninterrupted  operation  if  one  of  the 
power  supplies  fails.  When  all  power  supplies  are  functional,  they  should  share  the 
system  load  equally  by  operating  at  reduced  output.  If  the  system  power 
requirement  is  less  than  that  available  from  one  power  supply,  redundancy  should 
not  be  used  unless  a  critical  function  is  involved. 

Analysis  Recommendation:  Compliance  should  be  verified  by  electrical  loads 
analysis 

Test  Recommendation:  Not  applicable. 

(9)  Current  Sharing. 

Design  Guideline:  The  power  supplies  should  be  constructed  so  that  units  which 
have  the  same  output  voltage  may  operate  in  parallel.  The  design  should  be  such 
that  power  supply  failures  will  not  cause  degradation  of  parallel  power  supplies. 
Each  power  supply  should  provide  its  proportional  share  ( ±  in%)  of  the  total 
electric  load  required  at  the  configured  output  voltage. 

Analysis  Recommendation:  Compliance  with  the  specified  operation  should  be 
verified  as  a  part  of  the  design  process. 

Test  Recommendation:  A  demonstration  should  be  conducted  under  load  to 
verify  that  the  parallel  power  supplies  power  up  and  power  down  in  unison.  Failure 
and  reset  of  one  of  the  power  supplies  should  be  simulated  or  induced  to 
demonstrate  proper  operation  of  the  remaining  units  through  the  transition. 
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(10)  Protective  Device  Operation. 

Design  Guideline:  During  parallel  operation,  each  power  supply  protective 
device  should  be  capable  of  sensing  and  operating  independently  of  the  other 
power  supplies.  Master-slave  type  operation  should  not  be  p)ermitted  under  any 
circumstances. 

Analysis  Recommendation:  Compliance  with  the  specified  operation  should  be 
verified  as  a  part  of  the  design  process. 

Test  Recommendation:  A  demonstration  should  be  conducted  under  load  to 
verify  proper  operation  of  each  protective  device  during  parallel  operation 

e.  Memory  Fault  Tolerance 

(1)  Block  Masking. 

Design  Guideline:  Known  locations  of  defective  memory  should  be  mapped  out 
of  the  memory  directories.  In  this  manner,  permanently  failed  cells  can  be 
prevented  from  contributing  to  double  error  occurrences  in  combination  with  soft 
errors.  At  power-up  or  reinitialization,  BIT  should  perform  a  memory  test  routine 
and  leave  a  memory  map  of  all  good  blocks.  At  the  conclusion  of  the  memory  test 
routine,  all  words  contained  in  the  memory  blocks  marked  good  should  have  been 
initialized  in  an  error  free  data  pattern.  Program  loader  software  should  make  use 
of  the  good  memory  block  map,  the  processor  memory  mapping  registers,  and 
information  stored  in  program  file  headers  to  load  distributed  operating  syr  lems 
and  application  programs  into  the  remaining  good  areas  of  main  memory.  Repair 
or  replacement  of  the  module  should  not  be  required  until  the  number  of  remaining 
good  blocks  of  memory  are  insufficient  to  meet  operational  requirements. 

Analysis  Recommendation:  An  analysis  should  be  performed  to  identify  the 
optimum  combination  of  component/bit  mapping,  hardware  control,  and  software 
control. 

Test  Recommendation:  Not  applicable. 

(2)  Error  Detection/Correction. 

Design  Guideline:  As  a  minimum,  single  error  correct/double  error  detect  code 
should  be  used  in  large  bulk  semiconductor  memories.  It  should  be  considered  in 
any  application  involving  large  amounts  of  semiconductor  memory,  but  may 
impose  unacceptable  speed  and  complexity  penalties  in  some  applications  (e  g., 
CPU). 

Analysis  Recommendation:  A  detailed  timing  analysis  should  be  conducted  to 
determine  the  impact  of  this  technique  on  the  specific  application. 

Test  Recommendation:  System  bench  testing  should  be  used  to  insert  faults  and 
to  confirm  expected  system  operation. 
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5.1  Reliability  Demonstration  Testing.  This  appendix  presents  tables  and 
examples  which  summarize  the  following; 

•  MIL-HDBK-781  “Reliability  Design  Qualification  and  Production  Acceptance 
Tests;  Exponential  Distribution ...” 

•  Confidence  Interval  Calculations 

•  Poisson's  Exponential  Binomial  Limits 

5.2  MIL'HDBK-781  D  Test  Plans.  Tables  5. 1  and  5.2  summarize  standat d  test 
plans  as  defined  in  MIL-HDBK-781  D.  These  plans  assume  an  exponential  failure 
distribution.  For  nonexponential  situations  the  risks  are  different. 

The  fixed  length  test  plans  (Table  5.1 )  must  be  used  when  the  exact  length  and 
cost  of  the  test  must  be  known  beforehand  and  when  it  is  necessary  to 
demonstrate  a  specific  MTBF  to  a  predetermined  confidence  level  by  the  test  as 
well  as  reach  an  accept/reject  decision. 

The  probability  ratio  sequential  test  (PRST)  plans  (Table  5.2)  will  accept  material 
with  a  high  MTBF  or  reject  material  with  a  very  low  MTBF  more  quickly  than  fixed 
length  test  plans  having  similar  risks  and  discrimination  ratios.  However,  oitterent 
MTBF's  may  be  demonstrated  by  different  accept  decision  points  for  the  same  test 
plan  and  the  total  test  time  may  vary  significantly. 

Additional  guidance  on  test  plan  selection  is  provided  in  Section  T,  Topic  ^  5. 

5.2.1  Fixed  Length  Test  Plan  Example:  If  the  design  goal  MTBF  (Bo)  fc  a 
system  is  specified  as  750  hours  and  Test  Plan  XID  is  chosen,  the  follown  g 
statements  can  be  made: 

a.  There  is  a  20  percent  probability  of  rejecting  a  system  whose  true  MTBF  is  750 
hours  (producers  risk). 

b.  There  is  a  20  percent  probability  of  accepting  a  system  whose  true  MTBF  is 
500  hours  (consumers  risk). 

c.  The  lower  test  MTBF  (0,)  is  500  hours  (750/1.5). 

d.  The  duration  of  the  test  is  10,750  hours  (21.5  x  500). 

e.  The  test  will  reject  any  system  which  experiences  18  or  more  failures 

f.  The  test  will  accept  any  system  which  experiences  17  or  less  failures 

5.2.2  PRST  Test  Plan  Example:  If  the  design  goal  MTBF  (e^)  for  a  system  is 
specified  as  750  hours  and  Test  Plan  HD  is  chosen,  the  following  statements  can 
be  made: 

a.  There  is  a  20  percent  probability  of  rejecting  a  system  whose  true  MTBF  is  750 
hours  (producers  risk). 

b.  There  is  a  20  percent  probability  of  accepting  a  system  whose  true  MTBF  is 
500  hours  (consumers  risk). 

c.  The  lower  test  MTBF  (0,)  is  500  hours  (750/1.5). 
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d.  The  minimum  time  to  an  accept  decision  is  2095  hours  (4.19  x  500). 

e.  The  expected  time  to  an  accept  decision  is  5700  hours  (1 1.4  x  500).  (Expected 
time  to  decision  based  on  assumption  of  a  true  MTBF  equal  to  Bq.) 

f.  The  maximum  time  to  reach  an  accept  decision  is  10950  hours  (21.9  x  500). 
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Table  5.1 :  Fixed  Length  MIL-HDBK-781 D  Reliability  Demonstration  Test  Plans 
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5.3  Confidence  Level  Calculation  (Exponential  Distribution):  There  are  two 
ways  to  end  a  reliability  test,  either  on  a  specified  number  of  failures  occurring 
(failure  truncated),  or  on  a  set  period  of  time  (time  truncated).  There  are  usually 
two  types  of  confidence  calculations  of  interest,  either  one  sided  (giving  the 
confidence  that  an  MTBF  is  above  a  certain  value)  or  two  sided  (giving  the 
confidence  that  an  MTBF  is  between  an  upper  and  lower  limit).  The  four 
combinations  of  possibilities  are  summarized  by  the  chi-squared  models  given  in 
Table  5.3.  Each  of  the  possibilities  will  be  briefly  expanded  on. 


Table  5.3:  Summary  of  Chi-Squared  Models 


Two-Sided  Single-Sided 

Confidence  Confidence 

Level  Models  Level  Models 
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NoIm: 

C  =  number  of  failures  occurring  dunng  the  test 
a  =  risk  =  1  -  confidence  level 
»  =  point  estimate  MTBF  >  test  time/C 

=  chi-squared  statistical  distribution  value  from  Table  5  4  P  and  f  are  calculated  based  on  the  subscnpis  shown  in 
the  above  table  P  depends  on  the  confidence  interval  desired  and  f  depends  on  the  number  of  failures 
occurring 

CAUTION  Chi-Squared  confidence  interval  calculation  applies  only  to  systems  with  exponential  failure  dislnbulions 
Do  .  lOl  apply  to  M  TBCF 
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RELIABILITY  DEMONSTRATION  TESTING 


5.3.1  Failure  Terminated  Tests: 

5.3.11  Two*Sided  Confidence  Level  for  a  Test  Terminated  On  a  Failure.  Two- 
sided  confidence  levels  are  calculated  if  it  is  desired  to  put  both  upper  and  lower 
bounds  on  a  point  estimate  MTBF  (6). 

5.3.111  Failure  Terminated  (Two-Sided)  Example:  A  reliability  demonstration 
test  is  terminated  at  the  seventh  failure  which  occurs  at  820  hours  total  test  time. 
What  are  the  two-sided  80  percent  confidence  limits? 

Solution:  C  =  7 

e  =  820/7  =  11714 

Confidence  =  1  -  risk  =  1  -  a  =  .8 

c-  =  1  -  .8  =  .2 


From  Table  5.3: 

Simplifying: 
From  Table  5.4: 


2x7x117.14  2x7x117.14 

; - 

X  ,,  .  X  2.2.7 

\  2 

1639.96  ^  1639.96 

X  9.  14  X  1.  14 

x^  ,4  =  21.1  X^  14  =  7.79 


Calculating  the  Confidence  Limits:  «  e  «  16^^6 


77.7  «  e  «  210.5 

There  is  an  80  percent  probability  that  the  true  MTBF  6,  is  between  77.7  hours  and 
210.5  hours. 


5.3.12  One-Sided  Confidence  Level  for  a  Test  Terminated  on  a  Failure.  A 

majority  of  confidence  level  calculations  are  concerned  only  with  determining 
whether  a  minimum  level  of  MTBF  has  been  exceeded  with  a  certain  level  of 
confidence.  This  is  without  regard  to  what  the  upper  bound  may  be. 

5.3.12.1  Failure  Terminated  (One-Sided)  Example:  A  reliability  demoi^stration 
test  is  terminated  at  820  hours  total  test  time  after  the  occurrence  of  the  si^venth 
failure.  What  is  the  one-sided  lower  80  percent  confidence  limit? 

Solution:  C  =  7 

e  =  117.14 


Confidence  =  1  -  risk  =  1  -  a  =  .8 


a  =  0.2 


From  Table  5.3: 


2x7x  117.14 

X^1  -  2\  2-7 
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Simplifying: 


1639.96 
e.  14 


From  Table  5.4:  x^  e.,4  =  18.2 


Calculating  the  lower  confidence  level:  6 


1639.96 

18.2 


90.1 


There  is  an  80  percent  probability  that  the  true  MTBF,  6,  is  at  least  90.1  hours. 

5.3.2  Time  Terminated  Tests:  The  procedure  for  calculating  either  one  sided  or 
two  sided  confidence  intervals  for  a  time  terminated  test  is  analogous  to  the 
procedure  just  presented  for  the  failure  terminated  cases. 

5.4  Poisson  Distribution.  The  Poisson  distribution  is  useful  in  calculating  the 
probability  that  a  certain  number  of  failures  will  occur  over  a  certain  length  of  time 
for  systems  exhibiting  exponential  failure  distributions  (i.e.,  non-redundant 
systems).  The  Poisson  model  can  be  stated  as  follows: 


P(r)  = 


e  ’’(xt)' 
r! 


Where:  P(r)  =  probability  of  exacf/yr  failures  occurring 

X  =  the  true  failure  rate  per  hour  (i.e..  the  failure  rate  which  would  be 
exhibited  over  an  infinite  period) 

t  =  the  test  time 

r  =  the  number  of  failure  occurrences 

e  =  2.71828 .... 

!  =  factorial  symbol  (e.g.,  4!  =  4  x  3  2  x  i  ^  24, 0!  =  1,1!  =  1 ) 

The  probability  of  exactly  0  failures  results  in  the  exponential  form  of  this 
distribution  which  is  used  to  calculate  the  probability  of  success  for  a  given  period 
of  time  (i.e.,  P(0)  =  e The  probability  of  more  than  one  failure  occurring  is  the 
sum  of  the  probabilities  of  individual  failures  occurring.  For  example,  the  probability 
of  two  or  less  failures  occurring  is  P(0)  +  P(1)  -  P(2).  Table  5.5  is  a  tabulation  of 
exact  probabilities  used  to  find  the  probability  of  an  exact  number  of  failures 
occurring  Table  5.6  is  a  tabulation  of  cumulative  probabilities  used  to  find  the 
probability  of  a  specific  number  of  failures,  or  less,  occurring. 
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5.4.1  Poisson  Example  1 :  If  the  true  MTBF  of  a  system  is  200  hours  and  a 
reliability  demonstration  test  is  conducted  for  1000  hours,  what  is  the  probability  of 
accepting  the  system  if  three  or  less  failures  are  allowed? 

Solution;  Expected  number  of  failures  =  Xt  =  rjJsp  =  =  5 

M  I  or  2Q0 


From  Table  5.6,  the  probability  of  three  or  less  failures  (probability  of  acceptance) 
given  that  five  are  expected  is  .265.  Therefore,  there  is  only  a  26.5  percent  chance 
that  this  system  will  be  accepted  if  subjected  to  this  test. 

5.4.2  Poisson  Example  2:  A  system  has  an  MTBF  of  50  hours.  What  is  the 
probability  of  two  or  more  failures  during  a  10  hour  mission? 


Solution:  Expected  number  of  failures  = 


t 

MTBF 


50 


=  .2 


The  probability  of  two  or  more  failures  is  one  minus  the  probaDuny  of  one  or  less 
failures.  From  Table  5.6,  P(r^1)  when  .2  are  expected  is  .982. 

P(rs2)  =  1  -  P(r^1) 

1  -  .982  =  ,018 

Therefore,  there  is  a  very  remote  chance  (1.8  percent)  that  a  system  with  a  50  hour 
MTBF  will  experience  two  or  more  failures  during  a  10  hour  mission. 

5.4.3  Poisson  Example  3:  A  system  has  an  MTBF  of  50  hours.  What  is  Ihe 
probability  of  experiencing  two  failures  during  a  10  hour  mission? 

Solution:  Expected  number  of  failures  =  ~  ^  ^ 


From  Table  5.5,  the  probability  of  experiencing  exactly  two  failures  when  .2  are 
expected  is  .017  or  1.7  percent.  It  should  be  noted  that  the  probability  of 
experiencing  two  or  more  failures,  as  determined  in  the  last  example,  can  also  be 
determined  from  this  table  by  adding  P(r  =  2)  +  P(r  =  3)  when  .2  are  expected. 
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Table  5.6:  Summation  of  Terms  of  Poisson’s  Exponential  Binomial  Limit 


1000  times  the  probability  of  r  or  less  failure  occurrences  given  an  average  number  of  occurrences  equal  to  Xt. 
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247  592  833  946  986  997  999  1000 


223  558  809  934  981 


Table  5.6  (continued) 
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6.1  RGT  Definition.  MIL-STD-785  distinguishes  reliability  growth  testing  (RGT) 
from  reliability  qualification  testing  (RQT)  as  follows; 

Reliability  Growth  Test  (RGT):  A  series  of  tests  cortducted  to  disclose  deficiencies 
and  to  verify  that  corrective  actions  will  prevent  recurrence  in  the  operational 
inventory.  (Also  known  as  “TAAF"  testing). 

Reliability  Qualification  Test  (RQT):  A  test  conducted  under  specified  conditions, 
by,  or  on  behalf  of,  the  government,  using  items  representative  of  the  approved 
production  configuration,  to  determine  compliance  with  specified  reliability 
requirements  as  a  basis  for  production  approval.  (Also  known  as  a  "Reliability 
Demonstration,"  or  “Design  Approval”  test.) 

6.2  RGT  Application  Effectiveness.  An  effective  way  to  explain  the  concept  of 
RGT  is  by  addressing  the  most  frequently  asked  questions  relative  to  its  use  as 
summarized  from  "Reliability  Growth  Testing  Effectiveness"  (RADC*TR-84-20). 

For  more  information  consult  this  reference  and  MlL-HDBK-189,  “Reliability 
Growth  Management.” 

Who  pays  for  the  RGT?  Does  the  government  end  up  paying  more? 

The  usual  case  is  that  the  government  pays  for  the  RGT  as  an  additional  reliability 
program  cost  and  in  stretching  out  the  schedule.  The  savings  in  support  costs 
(recurring  logistics  costs)  exceed  the  additional  initial  acquisition  cost,  resulting  in 
a  net  savings  in  LCC.  The  amount  of  these  savings  is  dependent  on  the  quantity  to 
be  fielded,  the  maintenance  concept,  the  sensitivity  of  LCC  to  reliability  and  the 
level  of  development  required.  It  is  the  old  “pay  me  now  or  pay  me  later  situation" 
which  in  many  cases  makes  a  program  manager's  situation  difficult  because  his 
performance  is  mainly  based  on  the  "now”  performance  of  cost  and  schedule. 

Does  RGT  allow  contractors  to  “get  away  with”  a  sloppy  initial  design 
because  they  can  fix  it  later  at  the  governnient's  expense? 

It  has  been  shown  that  unforeseen  problems  account  for  75%  of  the  failures  due  to 
the  complexity  of  today's  equipment.  Too  low  an  initial  reliability  (resulting  from  an 
inadequate  contractor  design  process)  will  necessitate  an  unrealistic  growth  rate  in 
order  to  attain  an  acceptable  level  of  reliability  in  the  allocated  amount  of  test  time. 
The  growth  test  should  be  considered  as  an  organized  search  and  correction 
system  for  reliability  problems  that  allows  problems  to  be  fixed  when  it  is  least 
expensive.  It  is  oriented  towards  the  efficient  determination  of  corrective  action. 
Solutions  are  emphasized  rather  than  excuses.  It  can  give  a  nontechnical  person 
an  appreciation  of  reliability  and  a  way  to  measure  its  status. 

Should  all  development  programs  have  some  sort  of  growth  program? 

The  answer  to  this  question  is  yes  in  that  all  programs  should  analyze  and  correct 
failures  when  they  occur  in  prequalification  testing.  A  distinction  should  be  in  the 
level  of  formality  of  the  growth  program.  The  less  challenge  there  is  to  the  state-of- 
the-art,  the  less  formal  (or  rigorous)  a  reliability  growth  program  should  be.  An 
extreme  example  would  be  the  case  of  procuring  off-the-shelf  equipment  to  be  part 
of  a  military  system.  In  this  situation,  which  really  isn’t  a  development,  design 
flexibility  to  correct  reliability  problems  is  mainly  constrained  to  newly  developed 
interfaces  between  the  “boxes"  making  up  the  system.  A  rigorous  growrh  program 
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would  be  ina;  propriate  but  FRACAS  should  still  be  implemented.  The  other 
extreme  is  a  developmental  program  applying  technology  that  challenges  the 
state-of-the-art.  In  this  situation  a  much  greater  amount  of  design  flexibility  to 
correct  unforeseen  problems  exists.  Because  the  technology  is  so  new  and 
challenging,  it  can  be  expected  that  a  greater  number  of  unforeseen  problems  will 
be  surfaced  by  growth  testing.  All  programs  can  benefit  from  testing  to  find 
reliability  problems  and  correcting  them  prior  to  deployment,  but  the  number  of 
problems  likely  to  be  corrected  and  the  cost  effectiveness  of  fixing  them  is  greater 
for  designs  which  are  more  complex  and  challenging  to  the  state-of-the-art. 

How  does  the  applicability  of  reliability  growth  testing  vary  with  the 
following  points  of  a  development  program? 

(1 )  Complexity  of  equipment  and  challenge  to  state-of-the-art? 

The  more  complex  or  challenging  the  equipment  design  is,  the  more  likely 
there  will  be  unforeseen  reliability  problems  which  can  be  surfaced  by  a  growth 
program.  However,  depending  on  the  operational  scenario,  the  number  of 
equipments  to  be  deployed  and  the  maintenance  concept,  there  may  be  a  high 
LCC  payoff  in  using  a  reliability  growth  program  to  fine  tune  a  relatively  simple 
design  to  maximize  its  reliability.  This  would  apply  in  situations  where  the 
equipments  have  extremely  high  usage  rates  and  LCC  is  highly  sensitive  to 
MTBF. 

(2)  Operational  environment? 

All  other  factors  being  equal,  the  more  severe  the  environment,  the  higher  the 
payoff  from  growth  testing  This  is  because  severe  environments  are  more 
likely  to  inflict  unforeseen  stress  associated  reliability  problems  that  need  to  be 
corrected. 

(3)  Quantity  of  equipment  to  be  produced? 

The  greater  the  quantities  of  equipment,  the  more  impact  on  LCC  by  reliability 
improvement  through  a  reliability  growth  effort. 

What  reliability  growth  modei(s)  should  be  used? 

The  model  to  be  used,  as  MIL-HDBK-189  says,  is  the  simplest  one  that  does  the 
job.  Certainly,  the  Duane  is  most  common,  probably  with  the  AMSAA  (Army 
Materiel  Systems  Analysis  Activity)  second.  They  both  have  advantages;  the 
Duane  being  simple  with  parameters  having  an  easily  recognizable  physical 
interpretation,  and  the  AMSAA  having  rigorous  statistical  procedures  associated 
with  it.  MIL-HDBK-189  suggests  the  Duane  for  planning  and  the  AMSAA  for 
assessment  and  tracking.  When  an  RQT  is  required,  the  RGT  should  be  planned 
and  tracked  using  the  Duane  model;  otherwise,  the  AMSAA  model  is 
recommended  for  tracking  because  it  allows  for  the  calculation  of  confidence  limits 
around  the  data. 

Should  there  be  an  accept/reject  criteria? 

The  purpose  of  reliability  growth  testing  is  to  uncover  failures  and  take  corrective 
actions  to  prevent  their  recurrence.  Having  an  accept/reject  criteria  is  a  negative 
contractor  incentive  towards  this  purpose.  Monitoring  the  contractor  s  progress 
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and  loosely  defined  thresholds  are  needed  but  placing  accept/reject  criteria,  or 
using  a  growth  test  as  a  demonstration,  defeat  the  purpose  of  running  them.  A 
degree  of  progress  monitoring  is  necessary  even  when  the  contractor  knows  that 
following  the  reliability  growth  test  he  will  be  held  accountable  by  a  final  RQT.  Tight 
thresholds  make  the  test  an  RQT  in  disguise.  Reliability  growth  can  be  incentivized 
but  shouldn’t  be.  To  reward  a  contractor  for  meeting  a  certain  threshold  in  a  shorter 
time  or  by  indicating  "if  the  RGT  results  are  good,  the  RQT  will  be  waived,”  the 
contractor’s  incentive  to  "find  and  fix"  is  diminished.  The  growth  test’s  primary 
purpose  is  to  improve  the  design,  not  to  evaluate  the  design. 

What  is  the  relationship  between  an  RQT  and  RGT? 

The  RQT  is  an  "accounting  task”  used  to  measure  the  reliability  of  a  fixed  design 
configuration.  It  has  the  benefit  of  holding  the  contractor  accountable  some  day 
down  the  road  from  his  initial  design  process.  As  such,  he  is  encouraged  to 
seriously  carry  out  the  other  design  related  reliability  tasks.  The  RQT  is  an 
“engineering  task"  designed  to  improve  the  design  reliability.  It  recognizes  that  the 
drawing  board  design  of  a  complex  system  cannot  be  perfect  from  a  reliability  point 
of  view  and  allocates  the  necessary  time  to  fine  tune  the  design  by  finding 
problems  and  designing  them  out.  Monitoring,  tracking  and  assessing  the  resulting 
data  gives  insight  into  the  efficiency  of  the  process  and  provides  nonreliability 
persons  with  a  tool  tor  evaluating  the  development's  reliability  status  and  lor 
reallocating  resources  when  necessary.  The  forms  of  testing  serve  very  different 
purposes  and  complement  each  other  in  development  of  systems  and  equipments. 
An  RGT  is  not  a  substitute  for  an  RQT,  or  other  reliability  design  tasks. 

How  much  validity/confidenca  should  be  placed  on  the  numerical  results  of 
RGT? 

Associating  a  hard  reliability  estimate  from  a  grovyrth  process,  while  mathematically 
practical,  has  the  tone  of  an  assessment  process  rather  than  an  improvement 
process,  especially  if  an  RQT  assessment  will  not  follow  the  RGT.  In  an  ideal 
situation,  where  contractors  are  not  driven  by  profit  motives,  a  reliability  grov\rth  test 
could  serve  as  an  improvement  and  assessment  vehicle.  Since  this  is  noi  the  real 
world,  the  best  that  can  be  done  if  meaningful  quantitative  results  are  needed 
without  an  RQT,  is  to  closely  monitor  the  contractor  RGT  Use  of  the  AMSAA  model 
provides  the  necessary  statistical  procedures  for  associating  confidence  levels 
with  reliability  results.  In  doing  so,  closer  control  over  the  operating  conditions  and 
failure  determinations  of  the  RGT  must  be  exercised  than  if  the  test  is  for 
improvement  purposes  only.  A  better  approach  is  to  use  a  less  closely  controlled 
growth  test  as  an  improvement  technique  (or  a  structured  extension  of  FRACAS, 
with  greater  emphasis  on  corrective  action)  to  fine  tune  the  design  as  insurance  of 
an  accept  decision  in  an  RQT  With  this  approach,  monitoring  an  improvement 
trend  is  more  appropriate  than  development  of  hard  reliability  estimates.  Then  use 
a  closely  controlled  RQT  to  determine  acceptance  and  predict  operational  results, 

6.3  Duane  Model.  Because  the  Duane  model  is  the  one  most  commonly  used,  it 
will  be  further  explained.  The  model  assumes  that  the  plot  of  MTBF  versus  time  is 
a  straight  line  when  plotted  on  log-log  paper.  The  main  advantage  of  this  model  is 
that  it  is  easy  to  use.  The  disadvantage  of  the  model  is  it  assumes  that  a  fix  is 
incorporated  immediately  after  a  failure  occurs  (before  further  test  time  is 
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accumulated).  Because  fixes  are  not  developed  and  implemented  that  easily  in 
real  life,  this  is  rarely  the  case.  Despite  this  problem,  it  is  still  considered  a  useful 
planning  tool.  Below  is  a  brief  summary  of  the  Duane  model. 

a.  Growth  Rate  a  = 


b.  Cumulative  MTBF  MTBFc  = 


c.  Instantaneous  MTBF  MTBF,  = 


d.  Test  Time  T  = 

e.  Preconditioning  period  at  which  system  will  realize  an  initial  MTBF  of  MTBF^ 

Tpc  =  ^(MTBFpred) 

Where;  k  =  a  constant  which  is  a  function  of  the  initial  MTBF 
a  =  the  growth  rate 
T  =  the  test  time 

The  instantaneous  MTBF  is  the  model’s  mathematical  representation  of  the  MTBF 
if  all  previous  failure  occurrences  are  corrected.  Therefore,  there  is  no  need  to 
selectively  purge  corrected  failures  from  the  data. 

The  scope  of  the  up-front  reliability  program,  severity  of  the  use  environment  and 
system  state-of-the-art  can  have  a  large  effect  on  the  initial  MTBF  and,  theretore, 
the  test  time  required.  The  aggressiveness  of  the  test  team  and  program  office  in 
ensuring  that  fixes  are  developed  and  implemented  can  have  a  substantial  effect 
on  the  growth  rate  and,  therefore,  test  time.  Other  important  considerations  for 
planning  a  growth  test  are  provided  in  Table  6.1. 


AMTBF 

ATIME 


-T“ 

K  ' 


MTBFc 
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Table  6.1  RGT  Planning  Considerations 


•  To  account  for  down  time,  calendar  time  should  be  estimated  to  be 
roughly  twice  the  number  of  test  hours. 

•  A  minimum  test  length  of  5  times  the  predicted  MTBF  should  always  be 
used  (if  the  Duane  Model  estimates  less  time).  Literature  commonly 
quotes  typical  test  lengths  of  from  5  to  25  times  the  predicted  MTBF 

•  For  large  MTBF  systems  (e.g.,  greater  than  1000  hours),  the 
preconditioning  period  equation  does  not  hold;  250  hours  is  commonly 
used. 

•  The  upper  limit  on  the  growth  rate  is  .6  (growth  rates  above  .5  are  rare). 


6.4  Prediction  of  Reliability  Growth  Expected.  It  is  possible  to  estimate  the 
increase  in  reliability  that  can  be  expected  for  an  equipment  undergoing  a  reliability 
growth  development  program.  The  methodology  to  do  this  is  documented  in 
RADC-Tn-86-148  “Reliability  Growth  Prediction." 

6.4.1  Terms  Explained: 

\p  =  MIL-HDBK-217  predicted  equipment  failure  rate  (failures  per  hour). 

=  Equipment  rraturity  factor.  Estimated  as  me  pert-etiiage  of  the  design  which 
is  new. 

K,  =  Number  of  failures  in  the  equipment  prior  to  test. 

K,  =  30,000  X  F„  X  Xp 

Fa  =  Test  acce’nration  factor,  based  on  the  degree  to  which  the  test  environment 
cycle  represents  the  operational  environmental  cycle. 

P  ^  Toper ational  ^  Length  of  operational  life 
*'  Ttest  Length  of  test  cycle 


00005  ,p  . 
6.5 


6.4.2  Prediction  Procedure: 

a.  Calculate  the  equipment  MTBF  prior  to  test,  MTBF(o) : 

AATon,  V  f  0.0005  K,!  ’ 

MTBF(o)  =  [Xp  +  — g-g — J 

b.  Calculate  the  equipment  MTBF  after  "t”  hours  of  growth  testing: 


MTBF(t)  = 


Fa 

(Fa)(Xp)  +  K,  K.e--^^' 
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c.  Percent  MTBF  Improvement  =  ^  100 

6.4.3  Example: 

To  illustrate  application  of  the  reliability  growth  prediction  procedure,  consider  the 
following  hypothetical  example  of  an  avionics  equipment  to  be  subjected  'o 
reliability  growth  testing  during  full-scale  development.  The  following  assumptions 
are  made; 

•  40  percent  of  the  equipment  is  new  design;  the  remainder  is  comprised  of 
mature,  off-the-shelf  items. 

•  The  MIL-HDBK-217MTBF  prediction  is  300  hours  (\p  =  1/300). 

•  An  RGT  program  is  to  be  conducted  during  which  3000  hours  will  be 
accumulated  on  the  equipment. 

•  The  operational  cycle  for  the  equipment  is  a  ten-hour  aircraft  mission. 

•  The  test  profile  eliminates  the  period  of  operation  in  a  relatively  benign 
environment  (e.g.,  the  cruise  portion  of  the  mission)  resulting  in  a  test  cycle  of 
two  hours. 

The  predicted  number  of  failures  in  the  equipment  prior  to  testing  is: 

K,  =  30,000  X  (0.4)  X  (1/300)  =  40 
The  initial  MTBFis: 

MTBF(o)  =  [l/300  +  '  =  156  hours 

The  test  acceleration  factor  is; 

F*  =  10/2  =  5 

The  rate  of  surfacing  failures  during  the  test  is: 

=  (0.0005/6.5)  X  5  =  0.0003846 

The  equipment  MTBF  after  incorporation  of  corrective  actions  to  eliminatr?  those 
failures  identified  in  the  RGT  program  is: 

MTBF(3000)  =  5 '(5  x  1/300  +  40  x  0.0003846  e  o  «»3e46  -  3000)  =  232  hours 

Hence,  the  predicted  reliability  growth  is  from  an  initial  MTBF  of  156  hours  to  ar 
improved  MTBF  of  232  hours,  approximately  a  50  percent  improvement. 
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MAINTAINABILITY/TESTABILITY  DEMONSTRATION  TESTING 


7.1  Testing.  This  appendix  presents  a  listing  of  the  possible  maintainability 
demonstration  plans  as  determined  from  MIL-STD-471  “Maintainability  Verification 
Demonstration/Evaluation"  and  general  plans  for  testability  demonstrations.  In 
most  circumstances,  maintainability  and  testability  demonstrations  are  linked 
together  and  tested  concurrently.  Cor)current  testing  is  cost  effective  and  reduces 
the  total  number  of  tasks  that  must  be  demonstrated. 

7.2  Maintainability.  For  maintainability  there  are  two  general  classes  of 
demonstration;  tests  thai  use  naturally  occurring  failures,  and  tests  That  require 
induced  failures.  Natural  failure  testing  requires  a  long  test  period,  while  induced 
testing  is  only  limited  to  the  time  to  find  fix  the  fault.  To  run  a  thirty  task  test  using 
induced  faults,  the  test  time  should  be  less  than  a  week  while  a  natural  failure  test 
could  require  six  months  or  more  depending  on  the  failure  frequency. 

7.2.1  Maintainability  Test  Recommendations  (See  Table  7.1  for  complete  MIL- 
STD-471  Test  Plan  listing.) 

•  Test  plan  eight  should  be  used  if  dual  requirements  of  the  mean  and  either  90th 
or  95th  percentile  of  maintenance  times  are  specified  and  a  lognormal 
distribution  is  expected. 

•  Test  plan  nine  should  be  used  for  mean  corrective  maintenance,  mean 
preventive  maintenance  or  combination  of  corrective  and  preventive 
maintenance  testing.  Any  underlying  distribution  can  be  used  in  this  test  plan. 

•  The  sample  size  of  the  tasks  to  be  demonstrated  should  exceed  400  to  reduce 
the  risk  of  biasing  the  test  results. 

•  The  task  samples  must  be  based  on  the  failure  rate  distribution  of  the  equipment 
to  be  tested. 

•  Final  selection  of  the  tasks  to  be  demonstrated  must  be  performed  by  the 
procuring  activity  just  prior  to  test. 

7.3  Testability.  Three  parameters  which  are  usually  tested  In  a  testability 
demonstration  are:  the  fault  detection  capability,  the  fault  isolation  capability,  and 
the  false  alarm  rate.  Fault  detection  and  isolation  parameters  are  demonstrated 
using  induced  faults,  while  false  alarm  demonstrations  are  based  on  naturally 
occurring  events.  (See  Table  7.2  for  more  infoiiTiation  on  ieslability  demonstration.) 

7.3.1  Testability  Test  Recommendations. 

•  Fault  detection  and  isolation  testing  should  be  combined. 

•  Test  samples  should  exceed  400  to  reduce  any  bias 

•  The  test  samples  should  be  based  on  the  failure  rate  distribution  of  the 
equipment  to  be  tested. 

•  False  alarm  demonstration  should  be  a  data  collection  effort  using  all  the 
contractor  planned  tests  such  as  acceptance  testing  and  initial  operating  tests 
(lOT). 
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Table  7.1 ;  Maintainability  Demonstration  Test  Plan  Summary 
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Table  7.1  (continued) 
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Since  each  plan  demoi  istrates  a  different  testaointy  pararr  eier.  usual 
SeeMIL-STD-471A(page  78)  for  specific  demonstration  procedures 
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8.1  Reliability  Analysis  Center  (RAC) 

RAC  is  a  Department  of  Defense  Information  Analysis  Center  sponsored  by  the 
Defense  Logistics  Agency,  managed  by  the  Rome  Air  Development  Center 
(RADC),  and  currently  operated  at  RADC  by  IIT  Research  Institute  (IITRI).  RAC  is 
chartered  to  collect,  analyze  and  disseminate  reliability  information  pertaining  to 
electronic  systems  and  parts  used  therein.  The  present  scope  includes  integrated 
circuits,  hybrids,  discrete  semiconductors,  microwave  devices,  opto-electronics 
and  nonelectronic  parts  employed  in  military,  space  and  commercial  applications. 

Data  is  collected  on  a  continuous  basis  from  a  broad  range  of  sources,  including 
testing  laboratories,  device  and  equipment  manufacturers,  government 
laboratories  and  equipment  users  (government  and  non-government).  Automatic 
distribution  lists,  voluntary  data  submittals  and  field  failure  reporting  systems 
supplement  an  intensive  data  solicitation  program. 

Reliability  data  and  analysis  documents  covering  most  of  the  device  types 
mentioned  above  are  available  from  the  RAC.  Also,  RAC  provides  reliability 
consulting,  training,  technical  and  bibliographic  inquiry  services. 

For  Further  Technical  Assistance  and  Information  on  Available  RAC 
Services,  Contact: 

Reliability  Analysis  Center 

PO  Box  4700 

Rome  NY  13440-8200 

Technical  Inquiries;  (315)  330-9933 

Non-Technical  Inquiries:  (315)330-4151 

Autovon:  587-4151 

All  Other  Requests  Should  Be  Directed  to: 

Rome  Air  Development  Center 
RBE/Preston  R.  MacDiarmid 
Griffiss  AFB  NY  13441  -5700 
Telephone;  (315)330-7095 
Autovon:  587-7095 

8.2  Government  Industry  Data  Exchange  Program  (GIDEP) 

The  GIDEP  program  is  a  cooperative  activity  between  government  and  industry 
participants  for  the  purpose  of  compiling  and  exchanging  technical  data.  It 
provides  an  on-line  menu  driven  means  for  searching  for  desired  information. 
There  are  several  separate  data  banks  which  contain  R&M  related  information: 


Data  Bank 

Content 

Engineering 

Test  reports,  nonstandard  part 
justification  data,  failure  analysis  data, 
manufacturing  processes. 

RADC  RELIABILITY  ENGINEER  S  TOOLKIT 


A-77 


RELIABILITY  AND  MAINTAINABILITY  DATA  SOURCES 


Data  Bank 

Content 

Reliability  and  Maintainability  Failure  mode  and  replacement  rate  data 

on  parts,  reports  on  theories,  methods, 
techniques  and  procedures  related  to 
reliability  and  maintainability  practices. 

Failure  Experience  Failure  information  generated  on 

significant  problems  on  parts,  processes, 
materials,  etc.  Includes  ALERTS  and 
failure  analysis  information. 


GIDEP  provides  special  services  such  as  the  ALERT  system  which  notifies  all 
participants  of  significant  problem  areas  and  the  Urgent  Data  Request  System 
which  allows  all  participants  to  be  queried  for  information  to  solve  a  specific 
problem.  The  information  found  on-line  is  usually  a  brief  summary  of  a  report  or 
collected  data  which  provides  a  reference  for  further  detailed  information  found 
on  microfilm. 

For  further  information  on  how  to  access  and  use  GIDER  contact: 

GIDEP  Operation  Center 
Corona  CA  91720-5000 
(714)  736-4677 
Autovon;  933-4677 

8.3  Maintenance  and  Operational  Data  Access  System  (MODAS) 

MODAS  is  an  Air  Force  on-line  data  storage  and  access  system  to  track  field 
maintenance  events  for  purposes  of  product  improvement,  monitoring  product 
performance  and  enhancing  reliability  and  maintainability.  The  data  base  is  menu 
driven  and  contains  data  on  both  ground  and  airborne  equipment.  Data  can  be 
sorted  and  accessed  in  several  ways.  For  example,  data  on  the  top  50  most 
maintained  subsystems  on  an  aircraft  can  be  viewed  for  a  specific  geographical 
area  or  for  a  specific  aircraft  platform.  Mean-time-between-maintenance  actions 
(MTBMA)  can  be  calculated  from  the  data  on  airborne  systems  because  flight 
hours  are  also  provided  with  the  number  of  maintenance  actions.  For  further 
information  on  MODAS  content  and  access,  contact; 

Air  Force  Logistics  Command/MMTS 
Wright-Patterson  AFB  OH  45433 
(513)257-6906 
Autovon:  787-6906 
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8.4  Other  Data  Sources 
Air  Force 

Aircraft  Modification  Performance  Tracking  (AMPTS) 

AFLC/MME  (Perry  French) 

Autovon:  787-7886 
SA-ALC/MMEAI  (Herb  Cheesman) 

Autovon:  945-4225 

D041 :  Recoverable  Consumption  Item  Requirements  System 

AFLC/MMRS  (Sandra  Kirby) 

Autovon:  787-3460 

D066:  Commodity  Configuration  Management  System 

AFLC/MME  (Perry  French) 

Autovon:  787-7886 

D065  MAJCON  On-Line  Aerospace  Vehicle  Trainer  Reporting  System 

AFLC  (Elizabeth  Hayes) 

Autovon:  787-7705 

G311 :  Aerospace  Vehicle,  Inventory  and  Utilization 

HQ  USAF  (Mr.  Robinson) 

Autovon:  227-5405 

G021 :  Quality  and  Reliability  Assurance 

AFLC/QAA  (Jerry  Swanson) 

B456:  System  Effectiveness  Data  System  (SEDS) 

AFFIC/TEEES  (Jan  Howell) 

Autovon:  527-3066 

CDS:  F-16  Central  Data  Base 

Dynamics  Research  (Vern  Vutech) 

AFLC  LOC/TLPO  (Robert  DeSauty) 

Autovon:  787-5646 

Maintenance  Information  Logically  Analyzed  and  Produced  (MILAP) 

Langley  AFB  VA  (Sgt  Young) 

Automated  Maintenance  System  (AMS) 

WR-ALC  (Mike  Creasy) 

Robins  AFB  GA 

Navy 

3M:  Maintenance  Material  Management  (Avionics) 

NAMSO  (Manny  Pierucci) 

(717)  790-2031 

3M:  Maintenance  Material  Management  (Ships) 

NAMSO  (Larry  Costelac) 

(717) 790-7225 
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NALDA:  Naval  Aviation  Logistics  Data  Analysis  (AMPAS) 

NALC  Dept  NAM031  (Chuck  Carnobas) 

(800)  624-6621 

NALDA:  Naval  Aviation  Logistics  Data  Analysis  (FOJ) 

NALC  Dept  NAM031  (Chuck  Carnobas) 

(800)  624-6621 

QDEAS:  Quality  Deficiency  Evaluation  and  Analysis  System 

Navair  (Norman  Clark) 

(202)  422-4520 

AERMIP:  Aircraft  Equipment  Reliability  and  Maintainability 
Analysis  System 

NAVAIR  or  NAC 
Autovon:  724-7284 

Army 

Feedback  Analysis  Network  (FAN) 

AMCCOM  (Robert  Miller) 

(309)  782-2421 

Equipment  Improvement  Recommendations  (EIR) 

Part  of  FAN 

Equipment  Performance  Reports  (EPR) 

Part  of  FAN 

Quality  of  Deficiency  Reports 

Pari  of  FAN 

Sample  Data  Collection  Reports  (SDC) 

Part  of  FAN 

Test  Incident  Reports  (TIR) 

Pari  of  FAN 
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9.1  R&M  Education  Sources 

This  appendix  summarizes  the  following  R&M  educational  sources: 

•  DoD  Academic  Offerings 

•  Private  Institution  Academic  Offerings 

•  Short  Courses/Home  Study 

•  Periodicals 

•  Symposia  and  Workshops 

•  Textbooks 


9.2  DoD  Academic  Offerings 

Air  Force  Offerings: 

Air  Force  Institute  of  Technology  (AFIT)  School  ot  Systems  &  Logistics 
Wright-Patterson  AFB  OH  45433-6583 
Autovon:  785-6335/6336/3532 
Commercial:  (513)  255-6335/6336/3532 

The  Air  Force  Institute  of  Technology  offers  a  Systems  Engineering  Masters 
Degree  with  a  reliability  specialization.  They  also  offer  16  hours  of  engineering  post 
graduate  study  in  Professional  Specialized  Education  for  R&M,  as  well  as  an 
assortment  of  graduate  and  Professional  Continuing  Education  reliability  courses 
in  engineering  and  management. 

Titese  courses  are  primarily  for  DoD  personnel  but  if  certified  as  beneficial  to  the 
DoD,  contractor  personnel  may  attend  on  a  space  available  basis. 


Course  Title:  Reliability  and  Maintainability  Executive  Overview, 

QMT020  (JT) 

Length:  3  Class  Days 

Scope:  Basic  R&M  principles,  procedures  and  techniques  applied  to  design, 

development,  testing  and  management  are  explained  and  reiaieo  to 
the  system  acquisition  process,  life  cycle  cost  and  mission 
effectiveness. 


Course  Title: 

Length: 

Scope: 


Reliability  Centered  Maintenance  Analysis  (AFLC), 

LOG  032  (AF) 

5  Class  Days 

The  course  is  limited  to  the  AFLC  recommended  RCM  analysis 
procedures  and  oresented  only  on  site  at  AFLC  and  logistics  centers 


Course  Title:  Reliability  and  Maintainability  Design  in  Systems  Acquisition, 
QMT  335  (AF) 

Length:  10  Class  Days 
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Scope: 


Course  Title: 

Length: 

Scope: 


Course  Title: 

Length: 

Scope: 


Course  Title: 

Length: 

Scops: 


Provides  an  in-depth  explanation  of  the  principles,  procedures  and 
techniques  of  engineering  design  which  can  be  used  to  ensure  the 
development  of  reliable  and  maintainable  systems.  Design  related 
subjects  include  derating,  stress  analysis,  failure  modes  effect  and 
criticality  analysis,  sneak  circuit  analysis,  testability  and  accessibility. 
Achieving  R&M  through  test-analyze-and-fix,  combined  environment 
reliability  testing,  and  environmental  stress  screening  are  studied. 
The  course  also  includes  contracting  for  R&M  through  discussions  of 
requirements  development,  contractual  documentation,  source 
selection  and  post  contract  tasks.  Case  problems  and  workshops  are 
used  extensively  throughout  the  course  to  enable  the  student  to 
apply  the  theory  explained  in  the  classroom. 

Reliability,  QMT  372  (AF) 

15  Class  Days 

Includes  a  study  of  the  statistical  distributions  used  in  reliability 
including  the  binomial,  Poisson,  normal,  exponential  and  Weibull; 
reliability  allocation  and  prediction  techniques;  test  plans,  O.C. 
curves  and  the  use  of  military  standards;  data  analysis  and  the 
construction  and  interpretation  of  confidence  intervals;  applications 
of  mathematical  models;  reliability  program  management;  and 
current  problems  of  reliability.  The  participants  spend  the  last  week 
applying  these  principles  and  techniques  to  life  cycle  costing  in  a 
reliability  management  simulation  exercise. 

Mathematical  Statistics  and  Probability,  QMT  577  (AF) 

15  Class  Days 

An  educational  program  in  graduate  level  mathematical  statistics, 
probability  theory,  and  computer  programming.  Course  ncludes 
material  on  discrete  and  continuous  random  variables  and  their 
probability  distributions,  multivariate  probability  distributions, 
estimation,  hypothesis  testing,  and  order  statistics. 

Reliability  and  Maintainability  Research  and  Applications, 

QMT  578  (AF) 

15  Class  Days 

Includes  material  on  the  acquisition  process,  life  cycle  costing,  data 
management,  military  standards  and  handbooks  for  reliability  and 
maintainability,  simulation,  models,  reliability  and  maintainability 
management  and  quality  control.  Students  are  required  to  do 
research  on  an  R&M  program  and  spend  5  days  on  a  simulation 
exercise  performing  tasks  that  they  will  be  required  to  do  as  R&M 
engineers. 
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Course  Title:  Reliability  Theory,  QMT  579  (AF) 

Length:  20  Class  Days 

Scope:  A  graduate  level  course  \Arith  primary  emphasis  on  probabilistic 

engineering  design,  stress  strength  analysis,  Bayesian  methods, 
Weibull  applications,  estimation,  dynamic  reliability  models,  decision 
theory,  simulation,  and  recent  developments  in  reliability  and 
maintainability.  Several  typical  examples  will  be  presented  and 
discussed  to  illustrate  how  the  course  material  can  be  applied. 

Army  Offerings: 

U.S.  Army  Management  Engineering  Training  Activity  (AMETA) 

Director,  AMXOM/PMR 
Rock  Island  IL  61299-7040 
Autovon:  793-4041 
Commercial:  309-782-4041 

These  courses  are  primarily  for  DoD  personnel  but  if  certified  as  beneficial  to  the 

DoD,  contractor  personnel  may  attend  on  a  space  available  basis. 

Course  Title:  Reliability  and  Maintainability  Orientation  Seminar,  8A-F30  (JT) 

Length:  272  Days 

Scope:  The  course  provides  an  overview  of  the  R&M  activities  asso  lated 

with  each  of  the  life  cycle  phases  for  systems'equipment.  It 's  a  non¬ 
technical  course  based  on  DoD  Directive  5000.40.  DoD  R&M 
concepts  and  definitions,  engineering  activities,  accounting  ictivities 
appropriate  for  sound  decision  making,  and  management  activities 
are  discussed  along  with  the  relationship  of  reliability,  availability  and 
maintainability  to  factors  such  as  cost  and  logistics  support. 
Interrelationships  between  R&M,  and  disciplines  such  as 
configuration  management,  system  engineering,  logistics,  and 
procurement  are  portrayed.  Throughout  the  course  emphasis  is 
placed  on  successful  accomplishment  of  Governmental  R&M 
functions  as  distinguished  from  contract  responsibilities 

Course  Title:  Workshop  in  Reliability  &  Maintainability  Program 
Management,  7A-F28  (JT) 

Length:  2V2Days 

Scope:  Each  workshop  will  examine  high  visibility  lopics  which  are  of  current 

interest  to  the  R&M  program  manager.  The  format  and  content  of  the 
workshop  will  be  flexible  but  will  address  topics  with  the  goal  of 
developing  solutions  to  problems  of  immediate  interest  and  concern. 
Presentations  will  be  made  by  AMETA  faculty  and  recognized 
individuals  from  Government,  industry,  and  the  academic  community 
Attendees  should  be  prepared  for  active  participation  in  discussions 
of  R&M  applications  for  maximum  benefits  from  the  workshop. 


RADC  RELIABILITY  ENGINEER  S  TOOLKIT 


A-85 


RELIABILITY  AND  MAINTAINABILITY  EDUCATION  SOURCES 


Course  Title: 

Length: 

Scope: 


Course  Title: 

Length: 

Scope: 


Course  Title: 

Length: 

Scope: 


Special  Topics  in  Quality  and  Reliability,  AMETA-45 

1  -5  Days 

Course  content  is  fie^fible  and  may  include  modules  selected  from 
current  AMETA  course  offerings,  or  the  course  content  can  be 
specially  developed  material  for  a  particular  situation  or  group  of 
attendees.  The  type  of  educational  format  can  be  varied  to  fit  the 
subjects  selected  and  learning  objectives  of  the  class.  The  course 
configuration  will  be  based  upon  mutual  agreement  between  AMETA 
and  the  requesting  organization. 

Design  for  Reliability  and  Maintainability,  AMETA-118 

1  Week 

The  course  is  offered  as  three  separate  40  hour  modules.  All 
moduies  consider  design  from  an  ‘‘engineering”  perspective  and 
minimize  the  use  of  statistical  approaches.  Module  A  presents  topics 
associated  with  basic  reliability  techniques,  Module  B  presents  basic 
maintainability  techniques,  and  Module  C  presents  specialized 
design  techniques.  Topics  covered  in  each  module  are: 

Module  DRM  (A),  “Basic  Reliability  Design’’— (1 )  Overview  of 
the  Concept  of  Designing  for  Reliability;  (2)  Reliability  Modeling, 
Allocation  and  Prediction;  (3)  Failures  Modes,  Effects  and  Criticality 
Analysis  (FMECA),  Fault-tree  Analysis,  Cause-Consequence 
Diagrams;  (4)  Design  Techniques  (Operating  and  Stress);  (5)  Effects 
of  Functional  Testing,  Storage,  Handling;  and  (6)  Design  Review, 

Module  DRM  (B),  “Basic  Maintainability  Design’’— (1 )  Overview 
of  Maintainability  Design  Concepts;  (2)  Maintainability  Modeling, 
Allocation,  and  Prediction;  (3)  Equipment  Design  Guidelines;  (4) 
Automated  Diagnostics  (BIT,  BITE,  ATE);  and  (5)  Design  for 
Testability. 

Module  DRM  (C),  “Specialized  Reliability  Design’’— (1 ) 

Introduction  to  Specialized  Reliability  Design,  (2)  Sneak  Circuit 
Analysis,  (3)  Environmental  Stress  Analysis,  (4)  Thermal/Reliability 
Design  &  Analysis  Techniques,  (5)  Tolerance  Analysis,  and  (6) 
Additional  Special  Reliability  Design  Subjects — Electrostatic 
Discharge  Control  &  Software  Considerations. 

Software  Reliability  Test  and  Evaluation,  AMETA-120 

1  Week 

The  enrollee  will  be  exposed  to  material  that  provides  the  basis  for 
exploring  such  software  topics  as;  software  terminology  and  general 
applications,  DoD  acquisition  policy,  the  development  process, 
engineering  methods,  management  and  planning,  MIL-STD- 
52779A,  content  and  analysis  of  SQA  plans,  reviews  and  audits,  and 
future  trends.  Throughout  the  course,  emphasis  is  placed  on 
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successful  implementation  of  Governmental  reliability  and  quality 
software  development  programs.  Exercises  and  case  studies  are 
designed  to  give  students  an  opportunity  to  respond  to  actual 
situations. 

Course  Title:  Reliability  and  Maintainability  Engineering  Management, 
AMETA-101 

Length:  2  Weeks 

Scope:  This  is  a  core  course  in  the  R&M  fundamentals  which  are  necessary 

to  support  the  development  of  program  requirements  and  the 
administration  and  control  of  R&M  programs  throughout  the  life 
cycle.  Content  stresses  Government  R&M  responsibilities  in  the 
areas  of  establishing  and  overseeing  major  programs.  Coverage  will 
review  DoD  R&M  programs  and  policies,  address  tailoring  of  R&M 
program  requirements,  discuss  roles  and  relationships  of  functional 
groups,  and  identify  system  life  cycle  activities  having  significant 
impact  on  R&M.  In  addition,  R&M  engineering  and  accounting  tasks, 
practices  and  techniques  will  be  discussed  to  suggest  when,  how 
and  why  they  should  be  applied  in  a  given  program. 

Course  Title:  Production  Reliability  Assurance,  AMETA-100 

Length:  ?  Weeks 

Scope:  Topical  coverage  includes  an  introduction  to  and  a  brief  history  of 

reliability;  DoD  reliability  policy  and  definitions;  reliability  assurance 
in  the  production  process;  control  and  improvement  of  processes; 
the  importance  of  activities  such  as  quality  assurance,  systems 
engineering,  and  configuration  management;  probabilistic  and  non- 
probabilistic  testing;  and  management  and  control  of  production 
reliability  programs.  Coverage  addresses  Government  ar.d 
contractor  efforts  required  to  assure  product  and  replacement  part 
reliability  in  both  new  manufacture  and  rebuild  activities.  Formal 
assessment  is  made  of  the  enrollee's  mastery  of  the  course  content. 

Course  Title:  Reliability  and  Maintainability  Requirements,  Testing  and 
Evaluation,  AMETA-122 

Length:  2  Weeks 

Scope:  Content  of  the  course  will  include  modules  that  present,  consistent 

terminology  for  R&M  concepts  and  relationships,  the  methods  for 
determining  system  level  R&M  testing  and  how  to  conduct  R&M  test 
evaluations.  State-of-the-art  methodologies  will  be  presented  in 
lecture  and  practical  exercises. 

Course  Title:  Reliability  and  Maintainability  Testing,  8A-F27  (JT) 

Length:  2  Weeks 
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Scope:  Testing  concepts  and  definitions,  testing  and  screening  methods, 

R&M  tests  from  military  standards  and  handbooks,  and  selection  and 
application  of  specific  testing  and  screening  methods,  including 
shock,  vibration,  and  temperature  as  stress  parameters  are 
addressed.  System  level  testing  is  not  addressed  in  this  course. 

Course  Title:  Quality  and  Reliability  Assurance  Intern  Program,  AMETA-4 

Length:  26  Weeks 

Scope:  This  is  a  3-year  (maximum)  training  program  structured  in  three 

phases.  Phase  I  consists  of  formal  classroom  training  conducted  at 
AMETA  (26  weeks).  Phase  II  consists  of  OJT  involving  rotational 
assignments  at  the  interns  PDL  (66  weeks).  Phase  III  consists  of 
specialized  OJT  at  the  PDL  (64  weeks). 

Course  Title:  Army  R&M  Requirements,  AMETA-121 

Length:  2  Weeks 

Scope:  This  course  will  address  the  development,  optimization,  evaluation 

and  review  of  system  R&M  characteristics.  It  will  also  address  the 
establishment  and  testing  of  user  relevant  operational  R&M 
requirements.  (These  are  Minimum  Acceptable  Value  (MAV) 
requiremems.)  This  course  will  not  cover  the  engineering  methods 
that  are  used  to  determine  Best  Operational  Capability  (BOC) 
values. 

9.3  Private  Institution  Academic  Offerings 

The  University  of  Arizona  has  long  offered  a  Master  of  Science  degree  with  a 
reliability  engineering  option.  They  also  conduct  an  annual  five  day  Reliability 
Engineering  and  Management  Institute,  provide  short  courses  and  video  taped 
instruction.  Contact  Dr.  Dimitri  Kececioglu,  Aerospace  and  Mechanical 
Engineering  Department.  Bldg  16,  Rm  29CB,  University  of  Arizona,  Tucson  A2 
85721. 

The  University  of  Maryland  has  offered  a  Master  of  Science  in  Reliability 
Engineering  since  the  fall  of  1986.  They  also  provide  video  taped  instruction. 
Contact  Dr,  Marvin  L.  Roush,  Center  for  Reliability  Engineering,  Chemical  and 
Nuclear  Engineering  Building,  University  of  Maryland.  College  Park  MD  207'12 

The  New  Jersey  Institute  of  Technology  has  a  long  standing  graduate  program  in 
reliability  engineering.  Contact  Raj  Misra,  PhD,  Professor  of  Electrical  Engineering 
and  Reliability,  NJIT,  Newark  NJ  07102,  (201 )  596-351 1. 

Individual  courses  on  R&M  subjects  have  been  included  in  the  curricula  of  many 
schools,  including  Pennsylvania  State  University,  VPI,  USC,  Virginia  Tech,  SMU 
and  Syracuse  University.  There  are  probably  many  more. 
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9.4  Short  Courses/Home  Study 

Each  year,  many  short  courses  and  seminars  of  interest  to  the  R&M  community  are 
presented.  Most  presenters  will  arrange  to  bring  their  courses  to  any  specified 
location,  if  the  interest  is  sufficient. 

Besides  the  Universities  of  Arizona  and  Maryland,  frequent  offerings  of  short 
courses  on  Reliability  Engineering  topics  have  been  made  by  the  Short  Course 
Program  Office,  UCLA  Extension,  10995  LaConte  Ave,  Los  Angeles  CA  90024, 
and  The  Continuing  Engineering  Education  Center,  George  Washington  University, 
Washington  DC  20052. 

The  Reliability  Analysis  Center  (a  DoD  Information  Analysis  Center  managed  by 
RADC),  PO  Box  4700,  Rome  NY  13440,  offers  short  courses  in  Testability 
Practices  Today,  Statistical  Process  Control,  Design  Reliability,  and  Practical 
Statistical  Analysis  with  Reliability  Applications.  Contact  Mr.  S.  Flint  at  (315) 
330-4151. 

The  American  Society  for  Quality  Control  (ASQC)  provides  a  four  day  course  in 
Reliability  Engineering  as  well  as  a  variety  of  short  courses  on  Quality  Engineering 
and  Management,  including  one  on  Software  Quality  Assurance.  ASQC  also  offers 
home  study  courses,  video  tapes  on  selected  topics,  and  a  variety  of  textbooks.  An 
ASQC  Education  and  Training  Catalog  may  be  obtained  from  the  ASQC  Education 
and  Training  Institute,  310  West  Wisconsin  Ave,  Milwaukee  Wl  53203. 

The  IEEE  Reliability  Society  is  preparing  a  home  study  course  in  Reliability 
Engineering,  which  is  scheduled  for  completion  in  September  1987  and  release  in 
early  1988.  Availability  will  be  announced  in  the  IEEE  Spectrum.  Contact  Mr,  Henry 
Malec,  Digital  Equipment  Corp.,  550  King  Street,  Mailstop;  LKG  1-2/C12,  Littleton 
MA  01460-1289. 

9.5  Periodicals 

The  IEEE  Transactions  on  Reliability,  which  also  serves  as  the  journal  of  the 
Electronics  Division  of  ASQC,  is  published  five  times  a  year.  Subscriptions  are  free 
to  members  of  the  IEEE  Reliability  Society  and  the  Electronics  Division  of  ASQC. 
Others  may  obtain  copies  or  subscriptions  from  the  IEEE,  345  E.  47th  Street,  New 
York  City  NY  10017. 

Reliability  Review  is  published  by  the  Reliability  Division  of  ASQC.  It  is  free  to 
members.  Subscriptions  to  others  are  $20  per  year.  Contact  Reliability  Review, 
Subscriptions,  American  Society  for  Quality  Control,  310  W.  Wisconsin  Ave., 
Milwaukee  Wl  53203. 

Reliability  Engineering  is  a  monthly  international  journal  published  by  >ed 
Sciences  Publications  Ltd.,  Linton  Road,  Barking,  Essex,  England  iGlibJU. 
Subscriptions  can  be  obtained  from  Elsevier  Science  Publishing  Co.,  52  Vanderbilt 
Ave,  New  York  City  NY  10017.  Cost  is  $255  per  year. 

John  Wiley  &  Sons  Ltd,  Baffins  Lane,  Chichester,  Sussex  P0191 UD,  England 
publishes  the  Quarterly  Quality  and  Reliability  Engineering  International.  Current 
subscription  cost  is  $89  L'S  currency,  which  can  be  sent  to  Subscription  Dept  C, 
John  Wiley  &  Sons  Inc.,  605  Third  Ave,  New  York  City  NY  10158. 
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Evaluation  Engineering  Magazine  is  published  by  A.  Verner  Nelson  Associates, 
the  Nelson  Building,  1282  Old  Skokie  Rpad,  Highland  Park  IL  60035. 

News  of  current  activities  in  reliability  can  be  found  in  th^  R  aC  NEWSLETTER 
which  is  published  quarterly  and  distributed  free-of-charge  by  the  Reliability 
Analysis  Center,  RO.  Box  4700,  Rome,  NY  13440. 

The  RELIABILITY/MAINTAINABILITY  TECHNOLOGY  TRANSITION-FACT 
SHEET  is  published  twice  a  year  by  the  Rome  Air  Development  Center,  Systems 
Reliability  and  Engineering  Division.  A  letter  to  RADC/RBE-2,  Griffiss  AFB  NY 
13441  -5700  requesting  the  publication  is  all  that  is  required  for  inclusion  on  the 
regular  distribution  list. 

9.6  Symposia  and  Workshops 

The  Annual  Reliability  and  Maintainability  Symposium  is  the  premier  forum  for 
R&M  information  exchange.  Sponsored  by  10  professional  societies,  it  is  held 
every  year  in  January,  and  includes  tutorial  sessions.  For  further  information 
contact  IEEE  Reliability  Society,  7c  IEEE,  345  E.  47th  Street,  New  York,  NY  10017, 
(212)  705-7484  or  (212)  705-7900.  Back  copies  of  the  Symposium  Proceedings 
can  be  ordered  from  RAMS,  7c  Evans  Associates,  804  Vickers  Ave.,  Durham  NC 
27701. 

The  Annual  Reliability  Physics  Symposium  is  held  each  April  under  the 
sponsorship  of  the  IEEE  Reliability  and  Electron  Device  Societies.  For  further 
information,  contact  Dr.  Robert  W.  Thomas,  RADC/RBRE,  Griffiss  AFB  NY  13441  - 
5700.  Since  1985,  the  conference  has  been  recorded  on  video  tape,  available  from 
SAR  Associates,  RR  2-Box  500,  Rome  NY  13440. 

An  Annual  Spring  Reliability  Seminar  is  presented  by  the  Reliability  Society 
Chapter  of  the  IEEE  Central  New  England  Council.  The  most  recent  was  held  April 
1987,  and  the  contact  listed  was  Miss  Vivian  Thorsen,  Raytheon  Corporation 
(MET-5-1-210),  528  Boston  Post  Road.  Sudbury  MA  01776. 

The  Institute  of  Environmental  Sciences  holds  an  annual  workshop  on 
Environmental  Stress  Screening.  Reliability  issues  are  also  included  in  the  annual 
technical  meeting  and  equipment  exposition,  held  every  May.  Contact  lES  at  940 
East  Northwest  Highway,  Mount  Prospect  IL  60056,  (312)  255-1561. 

The  exchange  of  information  on  design,  engineering,  reliability,  and 
standardization  relative  to  microcircuit  applications  is  one  of  the  principal 
objectives  of  the  annual  Government  Microcircuit  Applications  Conference 
(GOMAC).  All  sessions  are  ITAR  controlled  and  one  session  is  classified  The 
conference  alternates  between  the  east  and  west  coasts  and  is  held  every 
October.  For  more  information  contact  Jay  Morreale  at  Palisades  Institute  for 
Research  Services,  Inc.,  201  Varick  Street,  11th  Floor,  New  York  NY  10014-4889  or 
Mark  Goldfarb,  Palisades  Institute  for  Research  Services,  Inc.,  Suite  307,  One 
Crystal  Park,  201 1  Crystal  Drive,  Arlington  VA  22202. 

Air  Force  Systems  Command  and  Air  Force  Logistics  Command  sponsor  a  joint 
R&M  Workshop  in  October  or  November  each  year.  The  workshop  is  usually  held 
at  Wright-Patterson  AFB  OH.  The  focal  point  for  the  conference  rotates  between 
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AFSC  and  AFLC.  For  more  information  contact:  (1 )  AFSC/PLE,  Andrews  AFB  DC 
20334-5000,  (301 )  981-6429  or  Autovon:  858-6429;  (2)  AFLC-MM,  Wright- 
Patterson  AFB  OH  45432.  (513)  257-2733  or  Autovon:  787-2733. 

9.7  Textbooks 

There  are  too  many  textbooks  on  Reliability  to  list  here.  A  broad  coverage  of  the 
technology  is  found  in  MIL-HDBK-338,  Electronic  Reliability  Design  Handbook, 
available  from  the  Naval  Publication  and  Forms  Center.  5801  Tabor  Ave, 
Philadelphia  PA  19120-5099. 


RADC  RELIABILITY  ENGINEER  S  TOOLKIT 


A-91 


Appendix  10 

R&M  Specifications,  Standards, 
Handbooks  and  RADC 
Technical  Reports 


RADC  RELIABILITY  ENGINEER  S  TOOLKIT 


A-93 


SPECS,  standards,  handbooks,  tech  reports 


10.1  Specifications,  Standards  and  Handbooks 

As  the  DoD’s  Lead  Standardization  Activity  for  the  Reliability  Standardization  Area 
(RELI)  and  the  Maintainability  Standardization  Area  (MNPv'),  RADC  publishes, 
every  two  years,  the  status  of  all  standardization  documents  in  each  of  these 
areas.  The  current  plans  are  Reliability  Standardization  Document  Program  Plan 
(Revision  4  dated  24  April  1987)  and  Maintainability  Standardization  Document 
Program  Plan  (Revision  2  dated  16  September  1985).  This  appendix  provides  a 
summary  of  military  documents  related  to  the  R&M  discipline.  Table  10.1  lists 
reliability  standards  and  handbooks  along  with  an  abbreviation  to  cross  reference 
the  custodial  agencies  which  are  listed  in  Table  10.3.  Table  10.2  lists  maintainability 
standards  and  handbooks  along  with  abbreviations  of  custodial  agencies  which 
are  listed  in  Table  10.3.  Table  10.4  lists  other  R&M  related  standards, 
specifications,  pamphlets  and  regulations. 
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SPECS.  STANDARDS.  HANDBOOKS.  TECH  REPORTS 
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Table  10.4  (continued) 
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10.2  R  ADC  Technical  Reports 

Table  10.5  summarizes  RADC  Technical  Reports  published  from  1975  through  this 
writing  which  are  related  to  R&M  design.  Documents  with  a  prefix  of  “A"  in  the  AD 
number  may  be  obtained  by  the  general  public  from; 

National  Technical  Information  Service 
Department  of  Commerce 
5285  Part  Royal  Road 
Springfield  VA  22151 
Phone:  (703)  487-4650 

United  States  Defense  contractors  registered  with  the  Defense  Technical 
Information  Center  may  obtain  RADC  reports  from: 

Defense  Technical  Information  Center 
Cameron  Station 
Alexandria  Station 
Alexandria  VA  22304 
Phone;  (202)  274-7633 
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Table  10.5  RADC  Reliability/Maintainability  Technical  Reports 


SPECS.  STANDARDS.  HANDB(X)KS.  TECH  REPORTS 
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79-327  AD-A082329  An  Objective  Printed  Circuit  Board  Testability  Design  and  Rating  System 


Table  10.5  (continued) 
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83-72  AD-A131358  The  Evolution  and  Practical  Applications  ol  Eailure  Mooes  ano  fcllects  Analyses 

83-85  AD- A 1 33624  Vol  I  Reliability  Programs  for  Nonelectronic  Designs 

AD  A  1 33625  VoMI 


83-108  AD-A 135705  Reliability  Modeling  of  Critical  Electronic  Devices 

83- 1 80  AD-A 138576  Condition  Monitoring  Techniques  for  Electromechanical  Equipment  Used  in  AF  Ground  C3I  Systems 

83-257  AD-A1 49683  Computer  Aided  Testability  Design  Analysis 

83-291  141147  Advanced  Applications  of  the  Printed  Circuit  Board  Testability  Design  and  Rating  System 

83-316  AD-B083630  Hardware/Software  Tradeoffs  for  Test  Systems 


SPECS,  STANDARDS.  HANDBOOKS.  TECH  REPORTS 
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86-97  AD-B1 084 1 7  Thermal  Resistances  of  Joint  Army  Navy  (JAN)  Certified  Microcircuit  Packages 
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SPECS.  STANDARDS.  HANOBOC  KS.  TECH  REPORTS 
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88-69  •  R/M/T  Ossign  for  Fault  Tolaranca  (T wo  Volumes) 

88-304  •  Reliability  Design  Criteria  for  High  Power  Tubes  (Three  Volumes) 
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MAJOR  AIR  FORCE  R&M  FOCAL  POINTS 


AF  Special  Atsittant  tor  RAM 
AF/LE-RO 

Wastiington  DC  20330-S130 
(202)  695-9e36/AV  225-9836 

«  Major  Command* 

*  Spacial  Operating  Agonciaa 

•  Direct  Reporting  Units 

Strategic  Air  Commend 
SAC'XPR 

Ottult  AFB  NE  68113 
(402)  294-5781 'AV  271-5781 

US  Space  Command 
SPACECMO'LKVY 
Peterson  AFB  CO  80914 
(303)  554-3286'AV  692-3286 

Elecironict  Security  Command 
ESC/LEMQ 

Kelly  AFB  TX  78243.5000 

(512)  925-2874 /AV  945-2874 
Pacilic  All  Force 
PACAF/DOOQ 

Hickham  AFB  HI  96853-5001 
(808)  449-6331 1 AV  449-6331 
Air  University 
AUXPO 

Maumrell  AFB  AL  36112-5001 
(205)  293-2159- AV  875-2159 

AF  Reserve  Headquaners 

AFRES  LGMAB 

Rooms  AFB  GA  31098 

(912)  926-3672.AV  468-3672 

Military  AidiX  (Command 

MAC'LG-R 

Scon  AFB  IL  62225 

(618)  256-404 1  ;AV  576-404 1 

AF  Systems  Command 
AFSC/PLE 

Andrews  AFB  DC  20334  5000 
(301)981-6429'AV  858-6429 

AF  Communicalions  Command 

AFCCrflE 

Scon  AFB  IL  62225 

(618)  256-4456'AV  576-4456 

At-  Coordinating  OH  for  Logistics 
AFCOLR'CC 

Wnghi-Patierson  AFB  OH  45433 

(513) 255-4758/AV  785-4758 

AF  Operational  Test  8  Evai  (Center 
AF0TEC'LG4W 
Kiriland  AFBNM87117 
(505)  846- 1296- AV  246-1296 

National  Guard  Bureau 
NGB'LGM 

INashinglon  DC  20310-2500 
(202)  695-0997  AV  225-0997 
Tactical  Air  Command 
TAC/SMORiM 
Langley  AFB  VA  23665 
(804)  764-72XrAV  574-7230 

AF  Logistics  Command 
AFLCrMM-R 

Wnght-Panerson  AFB  OH  45432 
(513)  257-5245/AV  787-5245 


Air  Force  R&M  Focal  Points 


AF  Logistics  Ops  Canter 
AFLC  LOC/CFE 

Wngtit-Patlarson  AFB  OH  45433 
(513)257-5567rAV:  787-5567 

AF  Acquisition  Logistics  Center 
AFALC'ER 

Wngtii-Patlerson  AFB  OH  45433 
(513)  225-2506/AV  785-2506 

AF  Tectinology  Appticalion  Center 

AFTACiAOE 

Patrick  AFB  FL  32925 

(305)  494-5175/AV  854-5175 

AF  Engmaenng  Services  Center 

AFESC'DEMM 

Tyndall  AFB  FL  32403 

(904)  283-6373'AV  523-6373 

Air  Training  Command 

ATC  LGX 

Randotph  AFB  TX  78150-5000 

(512)  652-2526'AV  487-2526 
Alaskan  Air  Command 

AAC  LGMM 

Elmendort  AFB  AK  99506-5001 
(907)  552-5606'AV  552-5606 
USAF  Europe 
USAFE  IXX3 
APO  NY  09012 
06371-47  AV  480-6568 

AF  Institute  ol  Tectinology 
AFIT  XP 

Wngni-Panerson  AFB  OH  45433 

(513) 255232I/AV  785-2321 

AF  Inspection  &  Safety  Cenier 
AFISC  SAMI 

Norton  AFB  CA  92409-7001 
(714)  382-2093' AV  876-2093 

AF  Military  Personnel  Center 
AFMPC'MPCYXX 
Randolpti  AFB  TX  78150 

(512)  6S2-5926'AV  487-5926 

•  AFSC  OivislctM 

Electronics  Systems  Division 
ESD  PL 

HansoomAFB  MA  01731 -5000 
(617)  271-6319'AV  478-5980 
X188-6319 

Aeronautical  Systems  Division 
ASO  EN 

Wnghl-Pstterson  AFB  OH  45433 

(513)  255-5874'AV  765-5874 

Space  Division 
SD  ALR 

PO  92960  LA  CA  90009-2960 
(213)643-1182'AV  833-1182 

Ballistic  Missile  Division 
BMO  AWR 

Norton  AFB  CA  92409-6468 
(714)  382-7067;AV  876-7067 

Armameni  Division 
AD' YIP 

Eglin  AFB  FL  32542-5000 
(904)  882-8652'AV  872-8652 


Space  &  Missile  Test  Orgamzstion 
ETR/ROM 

PatnckAFB  FL  32925 
(30SI494-4346/AV  854-4346 

Space  8  MissSe  Test  Organization 
WSMC/QA 

Vandanberg  AFB  CA  93437 
(805)866-6286/AV  276  6286 

AF  Contract  Mgmt  Division 
AFCMD/EPE 

KirVand  AFB  NM  87117-5000 
(505)  844-9518/AV:  244-9516 

AF  Flight  Test  Canter 
AFFTC'ENAR 
Edwards  AFB  CA  93523 
(805)  277-3066'AV  527-3066 

‘  AFSC  LabOfMoiiet 

Rome  Ak  Development  Center 
RAOC'RBE 

Gniliss  AFB  NY  13441  5700 
(315)  330-4921 /AV  587-4921 

AF  Armament  Laboratory 
AFATL/DLX 

Eglin  AFB  FL  32542-5434 
(904)  882  2205'AV  872  2205 

AF  INnght  Aeronautical  Labs 

afytalfix 

WPAFB  OH  45433 

(513)  255-5423/AV  785  5423 

Aerospace  Medical  Laboratory 

aamrltid 

WPAFB  OH  45433 

(513)  255-2423/AV  785-2423 

AF  INeapons  Laboratory 
AFWL/SU 

Kiriland  AFB  NM  871 17-6008 
(505)  244-9619'AV  244-9619 

AF  Engineer  SenntCi  Center 
AFESC'RDXP 
Tyndall  AFB  FL  32403 
(904)283-6288  AV  523-6288 

AF  Human  Resources  Lab 

AFHRL'LR 

WPAFB  OH  45433 

(513)  255-3713  AV  785-3713 

AF  Geophysics  Laboralory 
AFGL'XO 

Hanscom  AFB  M A  01 731 -5000 
(617)  861 -3006' AV  478-3006 
AF  Asironautical  Lab 
AFRPL  XR 

Edwards  AFB  CA  93523-5000 
(805)  277-5340'AV  525-5340 

AF  Space  Technology  Center 
AFSTC'XRN 

Kiriland  AFB  NM  871 1 7 -6008 
(515)B46-5545'AV  246-5545 
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ACRONYMS 


M- 

X 

a 

P 

Oc., 

e,-c 

e,.. 

e 

eo 

e, 

ep 

AAA 

ADAS 

ADM 

ADP 

ADPE 

AFALC 

AFLC 

AFR 

AFSC 

AFTO 

AGS 

Al 

AJ 

ALU 

AMGS 

AMSDL 

AP 

APTE 

APU 

ARM 

ASA 


Repair  Rate  (1/Mct) 

Failure  Rate  (1/MTBF) 

Producers  Risk 
Consumers  Risk 

Case  to  Ambient  Thermal  Resistance 
Junction  to  Case  Thermal  Resistance 
Junction  to  Ambient  Thermal  Resistance 
Observed  Point  Estimate  MTBF 
Upper  Test  (Design  Goal)  MTBF 
Lower  Test  (Unacceptable)  MTBF 
Predicted  MTBF 

Allocations,  Assessment  and  Analysis  (report) 

Architecture  Design  and  Assessment  System 

Advanced  Development  Model 

Automatic  Data  Processing 

Automatic  Data  Processing  Equipment 

Air  Force  Acquisition  Logistics  Center 

Air  Force  Logistics  Command 

Air  Force  Regulation 

Air  Force  Systems  Command 

Air  Force  Technical  Order 

Ambiguity  Group  Size 

Artificial  Intelligence 

Antijam 

Arithmetic  Logic  Unit 
Automatic  Microcode  Generation  System 
Acquisition  Management  Systems  and  Data  Control  List 
Array  Processor 

Automatic  Programmed  Test  Equipment 
Auxiliary  Power  Unit 
Antiradiation  Missile 
Advanced  Systems  Architecture 
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ACRONYMS 


ASD 

Aeronautical  Systems  Division 

ASIC 

Application  Specific  Integrated  Circuit 

ATE 

Automatic/ Automated  Test  Equipment 

ATF 

Advanced  Tactical  Fighter 

ATG 

Automatic  Test  Generation 

ATP 

Acceptance  Test  Procedure 

Ai 

Inherent  Availability 

Ao 

Operational  Availability 

B 

Bit 

BB,  B/B 

Brass  Board 

BCC 

Block  Check-Sum  Character 

BCS 

Bench  Check  Serviceable 

BIT 

Built-In-Test 

BITE 

Built  In  Test  Equipment 

BIU 

Bus  Interface  Unit 

BLER 

Block  Error  Rate 

BPS 

Bits  Per  Second 

C 

Centigrade 

C-ROM 

Control  ROM 

Command.  Control  and  Communications 

C=*CM 

Command.  Control,  and  Communications  Countermeasures 

C’l 

Command.  Control.  Communications  and  Intelligence 

CA 

Contracting  Activity 

CAD 

Computer  Aided  Design 

CAE 

Computer  Aided  Engineering 

CAM 

Content  Addressable  Memory 

CAS 

Column  Address  Strobe 

CASS 

Computer  Aided  Schematic  System 

CAT 

Computer  Aided  Test 

CB 

Chip  Boundary 

CCB 

Capacitive  Coupled  Bit 
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ccc 

Ceramic  Chip  Carrier 

CCD 

Charged  Coupled  Device 

CDR 

Critical  Design  Review 

CDRL 

Contract  Data  Requirements  List 

CFAR 

Constant  False  Alarm  Rate 

CGA 

Configurable  Gate  Array 

CIM 

Computer  Integrated  Manufacturing 

CISC 

Complex  Instruction  Set  Computer 

CIU 

Control  Interface  Unit 

CLCC 

Ceramic  Leaded  Chip  Carrier 

CML 

Current  Mode  Logic 

CMOS 

Complementary  Metal  Oxide  Semiconductor 

CND 

Can  Not  Duplicate 

CNI 

Communications,  Navigation,  and  Identification 

CODEC 

Coder  Decoder 

COMM 

Communications 

COMSEC 

Communications  Security 

COPS 

Complex  Operations  Per  Second 

CPM 

Control  Processor  Module 

CPU 

Central  Processing  Unit 

CRC 

Cyclic  Redundance  Check 

CS 

Chip  Select 

CSCI 

Computer  Software  Configuration  Item 

CSP 

Comrrion  Signal  Processor 

CSR 

Control  Status  Register 

CTE 

Coefficient  of  Thermal  Expansion 

CV 

Capacitance- Voltage 

D/A 

Digital-to-Analog 

dB 

Decibel 

dc 

Direct  Current 

DECTED 

Double  Error  Correcting,  Triple  Error  Detecting 

DED 

Double  Error  Detection 
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DESC 

DID 

DIP 

DISC 

DP 

DPA 

DRAM 

DUT 

DoD 

DoD-ADL 

EAROM 

ECC 

ECCM 

ECM 

EDA 

EDAC 

EDM 

EEPROM 

EEPROM 

EGC 

EGS 

EGSE 

EMC 

EMP 

EP 

EPROM 

ERPart 

ERC 

ESD 

ESD 

ESM 

ESS 
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Defense  Electronics  Supply  Center 
Data  Item  Description 
Dual  in-Line  Package 
Defense  Industrial  Supply  Center 
Data  Processor 
Destructive  Physical  Analysis 
Dynamic  Random  Access  Memory 
Device  Under  Test 
Department  of  Defense 
DoD  Authorized  Data  List 

Electrically  Alterable  ROM 
Error  Checking  and  Correction 
Electronic  Counter  Countermeasures 
Electronic  Counter  Measures 
Electronic  Design  Automation 
Error  Detection  and  Correction 
Engineering  Development  Model 
Electrically  Erasable  PROM 

Electrically  Erasable  Programmable  Read  Only  Memory 

Equivalent  Gate  Count 

Electronic  Ground  System 

Electronic  Ground  Support  Equipment 

Electromagnetic  Compatibility 

Electronic  Magnetic  Pulse 

Electrical  Parameter 

Erasable  PROM 

Established  Reliability  Part 

Electrical  Rule  Check 

Electronic  Systems  Division 

Electrostatic  Discharge 

Electronics  Support  Measure 

Environmental  Stress  Screening 
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ACRONYMS 


eV 

Electron  Volt 

EW 

Electronic  Warfare 

EXP 

Exponent 

F/W 

Firmware 

FAB 

Fabrication 

FAR 

False  Alarm  Rate 

FARR 

Forward  Area  Alerting  Radar  Receiver 

FBI 

Functional  Board  Test 

FD 

Fault  Detection 

FDl 

Fault  Detection  and  Isolation 

FFT 

Fast  Fourier  Transform 

FFTAU 

FFT  Arithmetic  Unit 

FFTCU 

FFT  Control  Unit 

FI 

Fault  Isolation 

FIFO 

First  In  First  Out 

FILO 

First  In  Last  Out 

FITS 

Failures  Per  10®  Hours 

FIT 

Fault  Isolation  Test 

FLIR 

Forward  Looking  Infrared 

FMC 

Full  Mission  Capability 

FMEA 

Failure  Modes  and  Effects  Analysis 

FMECA 

Failure  Modes,  Effects  and  Criticality  Analysis 

FOM 

Figure  of  Merit 

FP 

Floating  Point 

FPAP 

Floating  Point  Array  Processor 

FPLA 

Field  Programmable  Logic  Array 

FPMH 

Failures  Per  Million  Hours 

FQT 

Final  Qualification  Test 

FRACAS 

Failure  Reporting  and  Corrective  Action  System 

FRB 

Failure  Review  Board 

FS 

Full  Scale 

FSD 

Full  Scale  Development 
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ACRONYMS 


FSED 

Full  Scale  Engineering  Development 

FT 

Fourier  Transform 

GD 

Global  Defect 

GFE 

Government  Furnished  Equipment 

GIDEP 

Government  Industry  Data  Exchange  Program 

GM 

Global  Memory 

GOMAC 

Government  Microcircuit  Applications  Conference 

GSPA 

Generic  Signal  Processor  Architecture 

GaAs 

Gallium  Arsenide 

H/W 

Hardware 

HDL 

Hardware  Description  Language 

HDS 

Hierarchical  Design  System 

HFTA 

Hardware  Fault  Tree  Analysis 

HHDL 

Hierarchical  Hardware  Description  Language 

HOL 

Higher  Order  Language 

Hz 

Hertz 

1 

Current 

l/F 

Interface 

lAC 

Information  Analysis  Center 

lAW 

In  Accordance  With 

1C 

Integrated  Circuit 

ICT 

In  Circuit  Testing 

ICWG 

Interface  Control  Working  Group 

IDAS 

Integrated  Design  Automation  System 

lES 

Institute  of  Environmental  Sciences 

IFF 

Identification  Friend  or  Foe 

IFFT 

Inverse  FFT 

ILS 

Integrated  Logistics  Support 

IMPATT 

Impact  Avalanche  and  Transit  Time 

INEWS 

Integrated  EW  System 

ISA 

Instruction  Set  Architecture 
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ACRONYMS 


ISPS 

ITAR 

ITM 

JAN 

K 

KOPS 

LAN 

LCC 

LCCC 

LED 

LFR 

LHR 

LIF 

LIFO 

LISP 

LRM 

LRU 

LSA 

LSB 

LSE 

LSI 

LUT 

M 

M 

mA 

Met 

M-MM 

MAC 

MAP 

MBPS 

MCFOS 


Instruction  Set  Processor  Specification 
International  Traffic  In  Arms  Regulation 
Integrated  Test  and  Maintenance 

Joint  Army  Navy 

Thousand 

Kilo  Operations  Per  Second 

Local  Area  Network 
Life  Cycle  Cost 

Leadless  Ceramic  Chip  Carrier 
Light  Emitting  Diode 
Launch  and  Flight  Reliability 
Low  Hop  Rate 
Low  Insertion  Force 
Last  In  First  Out 
List  Processing 
Line  Replaceable  Module 
Line  Replaceable  Unit 
Logistics  Support  Analysis 
Least  Significant  Bit 
Lead  System  Engineer 
Large  Scale  Integration 
Look  Up  Table 

Maintainability 

Million 

Milliampere 

Mean  Corrective  Maintenance  Time 

Mean  Maintenance  Manhours 

Multiplier  Accumulator  Chip 

Modular  Avionics  Package 

Million  Bits  Per  Second 

Military  Computer  Family  Operating  System 
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ACRONYMS 


MCOPS 

MCTL 

MCU 

MDCS 

MDM 

MDT 

MDT 

MENS 

MENS 

MFLOPS 

MHz 

MIMIC 

MIPS 

MISD 

MLB 

MLIPS 

mm 

MMBF 

MMD 

MMH/FH 

MMH/PH 

MMM 

MMPS 

MMR 

MMS 

MMW 

MN 

MIN 

MNN 

MODEM 

MOPS 

MOS 


Million  Complex  Operations  Per  Second 
Military  Critical  Technology  List 
Microcontrol  Unit 

Maintenance  Data  Collection  System 
Multiplexer/Demultiplexer 
Maintainability  Demonstration 
Mean  Down  Time 
Mission  Element  Needs  Statement 
Mission  Equipment  Needs  Statement 
Million  Floating  Point  Operations  Per  Second 
Megahertz 

Microwave  Millimeter  Wave  Monolithic  1C 
Million  Instructions  Per  Second 
Multiple  Instructions  Single  Data 
Multilayer  Board 

Million  Logic  Inferencos/lnstructions  Per  Second 
Millimeter 

M3an  Miles  Between  FaHure 

Mean  Mission  Duration 

Maintenance  Manhours  Per  Flight  Hour 

Mean  Manhours  Per  Possessed  Hour 

Mass  Memory  Module 

Million  Multiples  Per  Second 

Multimode  Radar 

Mass  Memory  Superchip 

Millimeter  Wave 

Maintenance  Node 

Maintenance  Interface  Network 

Maintenance  Network  Node 

Modulator  Demodulator 

Million  Operations  Per  Second 

Metal  Oxide  Semiconductor 
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ACRONYMS 


MOSFET 

MP 

MPCAG 

MRAP 

ms 

MSB 

MSI 

MTBCF 

MTBD 

MTBUE 

MTBF 

MTBFF 

MTBM-IND 

MTBM-INH 

MTBM-ND 

MTBM-P 

MTBM-TOT 

MTBMA 

MTBR 

MTBUMA 

MTE 

MTE 

MTI 

MTTE 

MTTF 

MUX 

MW 

MWPS 

Mb 

Mil 

NOT 

NMOS 


Metal  Oxide  Semiconductor  Field  Effect  Transistor 

Maintenance  Processor 

Military  Parts  Control  Advisory  Group 

Microcircuit  Reliability  Assessment  Program 

Millisecond 

Most  Significant  Bit 

Medium  Scale  Integration 

Mean  Time  Between  Critical  Failures 

Mean  Time  Between  Demand 

Mean  Time  Between  Downing  Events 

Mean  Time  Between  Failure 

Mean  Time  Between  Functional  Failure 

Mean  Time  Between  Maintenance-Induced  (Type-2  Failure) 

Mean  Time  Between  Maintenance-Inherent  (Type-1  Failure) 

Mean  Time  Between  Maintenance-No  Defect  (Type-6  Failure) 

Mean  Time  Between  Maintenance-Preventive 

Mean  Time  Between  Maintenance-Total 

Mean  Time  Between  Maintenance  Actions 

Mean  Time  Between  Removals 

Mean  Time  Between  Unscheduled  Maintenance  Actions 

Multipurpose  Test  Equipment 

Minimal  Test  Equipment 

Moving  Target  Indicator 

Mean  Time  To  Error 

Mean  Time  To  Failure 

Multiplexer 

Milliwatt 

Million  Words  Per  Second 
Megabit 

1000th  of  an  inch 

Nondestructive  Testing 

N-Channel  Metal  Oxide  Semiconductor 
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ACRONYMS 


ns 

Nanosecond 

OPS 

Operations  Per  Second 

OROM 

Optical  ROM 

OTS 

Off-The-Shelf 

RM 

Programmable  Array  Logic 

PAT 

Programmable  Alarm  Thresholds 

PC 

Printed  Circuit 

PCB 

Printed  Circuit  Board 

PD 

Power  Dissipation 

PDL 

Program  Design  Language 

PDR 

Preliminary  Design  Review 

PGA 

Pin  Grid  Array 

PLA 

Programmable  Logic  Array 

PLCC 

Plastic  Leadless  Chip  Carrier 

PLD 

Programmable  Logic  Device 

PMD 

Program  Management  Directive 

PMOS 

P-Channel  Metal  Oxide  Semiconductor 

PMP 

Program  Management  Plan 

PMP 

Parts,  Materials  and  Processes 

PPB 

Parts  Per  Billion 

PPM 

Parts  Per  Million 

PPSL 

Preferred  Parts  Selection  List 

PRR 

Production  Readiness  Review 

PRST 

Probability  Ratio  Sequential  Test 

PW 

Pulse  Width 

PWB 

Printed  Wiring  Board 

Poly 

Polycrystalline  Silicon 

QA 

Quality  Assurance 

QC 

Quality  Control 

QPL 

Qualified  Parts  List 

QUMR 

Quality  Unsatisfactory  Material  Report 
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ACRONYMS 


R 

Reliability 

R&M 

Reliability  and  Maintainability 

RAD 

Measurement  of  Radiation 

RAM 

Random  Access  Memory 

RAMS 

Reliability  and  Maintainability  Symposium 

RD 

Random  Defect 

RDT 

Reliability  Demonstration  Test 

REG 

Register 

RETOK 

Retest  Okay 

RFP 

Request  For  Proposal 

RISA 

Reduced  Instruction  Set  Architecture 

RISC 

Reduced  Instruction  Set  Computer 

RIW 

Reliability  Improvement  Warranty 

RMS 

Remote  Monitoring  System 

RMS 

Root  Mean  Square 

ROM 

Read  Only  Memory 

ROT 

Reliability  Qualification  Test 

RSA 

Rapid  Simulation  Aids 

RSR 

Runtime  Status  Register 

RTL 

Register  Transfer  Language 

RTOK 

Retest  Okay 

RTQC 

Real  Time  Quality  Control 

Rads(Si) 

Rads  Silicon  (Total  Dose) 

S/N 

Signal  to  Noise  Ratio 

s/w 

Software 

SAMSO 

Space  and  Missile  Systems  Organization 

SAR 

Synthetic  Aperture  Radar 

SAW 

Surface  Acoustic  Wave 

SCD 

Specification  Control  Drawing 

SD 

Space  Division 

SDI 

Strategic  Defense  Initiative 

SDL 

System  Description  Language 
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ACRONYMS 


SDS 

Structured  Design  System 

SE 

Support  Equipment 

SECDED 

Single  Error  Correction,  Double  Error  Detection 

SED 

Single  Error  Detection 

SEM 

Standard  Electronic  Module 

SER 

Soft  Error  Rate 

SEU 

Single  Event  Upset 

SIP 

Single  In-Line  Package 

SMD 

Standard  Military  Drawing 

SMD 

Surface  Mounted  Device 

SMT 

Surface  Mounted  Technology 

SOI 

Silicon  On  Insulator 

SOIC 

Small  Outline  1C 

SOS 

Silicon  On  Sapphire 

SOW 

Statement  of  Work 

SPAD 

Scratch  Pad  Memory 

SR 

Slew  Rate 

SRA 

Shop  Replaceable  Assembly 

SRAM 

Static  RAM 

SRAP 

Semiconductor  Reliability  Assessment  Program 

SRL 

Shift  Register  Latch 

SRR 

Systems  Requirements  Review 

SRU 

Shop  Replaceable  Unit 

SRU 

Shop  Repairable  Unit 

SSEB 

Source  Selection  Evaluation  Board 

SSI 

Small  Scale  Integration 

SSPA 

Submicron  Signal  Processor  Architecture 

ST 

Self  Test 

STE 

Special  Test  Equipment 

TBD 

To  Be  Determined 

TC 

Temperature  Coefficient 

TCE 

Thermal  Coefficient  of  Expansion 

A-124 


RADC  RELIABILITY  ENGINEER  S  TOOLKIT 


TCR 

TDM 

TET 

TM 

TMDE 

TMP 

TPS 

TRD 

TRR 

TSMD 

TTL 

Ta 

Tj 

Tstg 

UHF 

ULSI 

UMF 

UUT 

V 

VCP 

VHDL 

VHSIC 

VIA 

VIM 

VLSI 

VSM 

VSP 

VTB 

WAM 

WSI 

WSIC 


ACRONYMS 

Temperature  Coefficient  of  Resistance 
Time  Division  Multiplexing 
Technical  Evaluation  Team 
Test  Modules 

Test  Measurement  and  Diagnostic  Equipment 

Test  and  Maintenance  Processor 

Test  Program  Set 

Test  Requirements  Document 

Test  Readiness  Review 

Time  Stress  Measurement  Device 

Transistor-Transistor  Logic 

Ambient  Temperature 

Junction  Temperature 

Storage  Temperature 

Ultra  High  Frequency 
Ultra  Large  Scale  Integration 
Universal  Matched  Filter 
Unit  Under  Test 

Volt 

VHSIC  Communications  Processor 
VHSIC  Hardware  Description  Language 
Very  High  Speed  Integrated  Circuit 

Interconnection  point  between  different  metal  conduction  layers. 

VHSIC  Insertion  Module 

Very  Large  Scale  Integration 

VHSIC  Submicron 

Variable  Site  Parameters 

VHSIC  Technology  Brassboard 

Window  Addressable  Memory 
Wafer-Scale  Integration 
Wafer  Scale  Integrated  Circuit 
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ACRONYMS 


X  Reactance 

Y  Admittance 

Z  Impedance 


A-126 


RADC  RELIABILITY  ENGINEER  S  TOOLKIT 


MISSION 


Rome  Air  Development  Center 


RADC  plans  and  executes  research,  development .  test  arid 
selected  acquisition  programs  in  support  of  Command,  Control, 
Communications  and  Intelligence  (C^I)  activities  Technical  and 
engineering  support  within  areas  of  competence  is  provided  to 
ESD  Program  Offices  (POs)  and  other  ESD  elements  to 
perform  effective  acquisition  of  C^I  systems  The  areas  of 
technical  competence  include  communications,  command  and 
control,  battle  management  information  processing,  surveillance 
sensors,  intelligence  data  collection  and  handling,  solid  state 
sciences,  electromagnetics,  and  propagation,  and  electronic 
reliability /maintainability  and  compatibility. 
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